Use ChainRules for operators
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Refactor
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- julia
- Domain
- machine-learning
Research direction
Start by reading src/Nonlinear/univariate_expressions_generator.jl and the operator dispatch around src/Nonlinear/operators.jl:570-582. Trace how ReverseAD represents classical, registered, and user-defined operators, then compare that flow with the proposed ChainRules-style dispatch and fallback. Done requires a decided design, updated operator handling, and evidence that the supported cases avoid the current type-instability concerns.
Written by the indexing model from the issue text.
Description
Currently, the approach in ReverseAD is to generate the symbolic expression of the first and second-order derivatives for classical univariate functions using Calculus
https://github.com/jump-dev/MathOptInterface.jl/blob/100eab2e669e73689e1dc214391d97c24402e35c/src/Nonlinear/univariate_expressions_generator.jl
Then, given a representation of the operator as an Int, we do an hard-coded binary search to evaluate a O(log(n)) number of Int comparison instead of a O(n) number of comparison:
https://github.com/jump-dev/MathOptInterface.jl/blob/100eab2e669e73689e1dc214391d97c24402e35c/src/Nonlinear/operators.jl#L570-L582
I'm wondering whether we could get closer to ChainRules instead like other Julia AD framework.
The naive way to do this would be
op = :tanh
f = eval(op)
value_and_derivative(f, 1)
The issue is that, because the value of op is discovered at run-time, the type of f is type-unstable.
But we can use the same trick with the if-else and do
if op == :tanh
value_and_derivative(tanh, x)
elseif op == :tan
value_and_derivative(tan, x)
elseif ...
else
value_and_derivative(eval(op), x)
end
Again, we can do a binary search instead of just a list of if-else.
So, for a fixed number of symbols, we avoid the type-instability thanks to the if-else and we have a fallback for the other ones with the eval.
That would also mean that for registered functions, we need to implement a method and just rely on multiple dispatch instead of adding an operators to the list of user-defined operators, user-defined operators already trigger a type-instability when they are called anyway.
- Dominant language
- Julia
- Stars
- 6
- Forks
- 0
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from blegat/ArrayDiff.jl
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
blegat/ArrayDiff.jl#83 ·
-
Building matrix Open
Difficulty 3/5 1-2 days Newbie friendliness 45/100
blegat/ArrayDiff.jl#13 ·
All issues in blegat/ArrayDiff.jl
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
epiforecasts/BVDOutbreakSize#862 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
CliMA/Oceananigans.jl#6066 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
JuliaLang/LinearAlgebra.jl#1707 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
JuliaPluto/Pluto.jl#3616 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
ReactiveBayes/ExponentialFamily.jl#322 · 2 comments ·