Supporting Linear Algebraic Primitives
还没有人认领这个 Issue。
评估
调研方向
首先查看现有的标量 DiffRules 接口以及相关的 DiffLinearAlgebra 讨论，其中包括拟议的张量/标量区分、原地方法和伴随变量。该 issue 未指定文件或测试；完成这项工作需要确定线性代数原语的接口和实现计划，以及它们的 forward-mode 和 reverse-mode 规则。
由索引模型根据 Issue 内容生成。
描述
(continued from https://github.com/invenia/Nabla.jl/issues/81, cc @willtebbutt)
This means that we would need a slightly more general interface for linear algebra, and would certainly require different forward- and reverse- mode expressions, than is currently provided by DiffRules.
Agreed, DiffRules only properly handles scalar kernels now. To support linear algebra, we need to add a notion of tensor/scalar, allowing in-place methods, marking adjoint variables, etc. to DiffRules.
Regarding the where possible statement above, there are certainly operations (such as the Cholesky factorisation) for which symbolic expressions might be a bit unweildy (see here). Now that I think about this a bit more, I'm not sure whether these larger implementations are going to be problematic or not.
Anyway, my point is that given symbolic expressions for the linear algebra operations I agree that it's reasonable to hope that compiler optimisations can eliminate redundant code when compiling custom kernel implementations, and that this is a significantly better idea than hand-coding lots of optimisations. (The issue I linked in my previous comment is a good example of this. I would definitely prefer to be able to just forget about this). However, I would contend that you simply can't handle linear algebra properly without a number of hand-coded symbolic expressions for the forward- and reverse-mode sensitivities because they aren't written in Julia. If at some point in the future we have native Julia implementation of (for example) LAPACK, then it would be a really good idea to try and produce an AD tool which is able to produce reasonably-well optimised kernels for each operation. To the best of my knowledge, we shouldn't expect this to happen any time soon (and almost certainly never for BLAS), so a symbolic version of the current implementation of DiffLinearAlgebra will be necessary for Capstan to be able to differentiate arbitrary Julia code even reasonably efficiently.
I think there might've been a misunderstanding with my previous post 😛I definitely am not arguing that we should express e.g. complex LAPACK kernels symbolically, and I didn't mean to imply that DiffRules/DiffLinearAlgebra were directly competing approaches. On the contrary, I think they're quite complementary - if DiffLinearAlgebra didn't exist, I eventually would need to make a "DiffKernels.jl" anyway. DiffRules is useful for mapping primal functions to derivative functions, and is thus useful when generating e.g. instruction primitives/computation graphs within downstream tools (i.e. it solves the problem "what kernels should I call and how should I call them?"). DiffLinearAlgebra (as it stands) is useful for providing implementations of these kernels (i.e. solves the problem "how do I execute the kernels that I'm calling?"). They're both necessary components of the AD ecosystem.
As for deciding what computations should be primitives, I think we're already on the same page; a computation should be defined as a primitive if either/both of the following applies:
- it is difficult to express the computation as a composition of existing primitives
- a hand-optimized kernel for the computation is sufficiently more performant than the equivalent composition of existing primitives, even after taking into account potential compiler-level optimizations.
- 主要语言
- Julia
- 星标
- 74
- 派生
- 36
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
JuliaDiff/DiffRules.jl 的其他 Issue
-
Package extensions可能已有人在做 @devmotion 于 29 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 48/100
JuliaDiff/DiffRules.jl#106 · 2 条评论 · 2 个 reaction ·
-
难度 3/5 1-2 天 新手友好度 35/100
JuliaDiff/DiffRules.jl#104 ·
-
难度 4/5 3-5 天 新手友好度 35/100
JuliaDiff/DiffRules.jl#102 · 2 条评论 ·
-
Problems with the `DEFINED_DIFFRULES` implementation可能已有人在做 @devmotion 于 29 天前认领。 未关闭
难度 5/5 一周以上 新手友好度 25/100
JuliaDiff/DiffRules.jl#90 · 1 个 reaction ·
-
难度 5/5 一周以上 新手友好度 25/100
JuliaDiff/DiffRules.jl#68 · 2 条评论 ·
查看 JuliaDiff/DiffRules.jl 的全部 Issue
相似的 Issue
-
R_to_gamma and R_to_v are wrong for |Z| != 1 (e.g. heavy ions)可能已有人在做 @DavidSagan 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
bmad-sim/BeamTracking.jl#366 ·
维护者通常 1 天内回复
-
TSDAE's docstring implies a theta argument it does not take可能已有人在做 @singhharsh1708 今天认领。 未关闭documentation
难度 1/5 1 小时以内 新手友好度 74/100
SciML/PETScDiffEq.jl#282 ·
维护者通常 1 天内回复
-
enhancement good first issue help wanted
难度 2/5 半天 新手友好度 82/100
-
SavedSubsystem with an integer save_idxs maps the index the wrong way round可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
SciML/SciMLBase.jl#1643 ·
维护者通常 2 天内回复
-
authority-vechtstromen ready-for-agent
难度 1/5 1 小时以内 新手友好度 75/100
Deltares/Ribasim-NL#890 ·
维护者通常 1 天内回复