User defined gradients
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
The issue does not name specific files, tests, or entry points. Start by locating the existing user-defined function and autodiff mechanisms, then determine how a function could provide both its value and gradient while reusing intermediate values. Done means a documented API and tests showing that iterative algorithms avoid unnecessary autodiff work.
Written by the indexing model from the issue text.
Description
This has been come up a few times, and we discussed this yesterday
If a user defined function includes iterative algorithm with a fixed point the implict gradient is 0, but autodiff will diff through that iterative algorithm which may significant cost per each iteration. In such cases it would be useful if the user could define the gradient computation. If that gradient computation could be part of the user defined function (e.g. returning a tuple of value and gradient), then the value and gradient could be computed using the same intermediate values. This would be useful in cases, where the user defined function is something which would not be likely to be added to Stan math, but where avoiding the autodiff through iterative part could make the gradient computation order of magnitude faster.
It's not clear how complicated this would be to implement, but it was suggested that I create this issue and tag @SteveBronder
- Dominant language
- C++
- Stars
- 2.8k
- Forks
- 388
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 15
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stan-dev/stan
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
-
Difficulty 3/5 1-2 days Newbie friendliness 66/100
-
algorithm code cleanup help wanted
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
good first issue i/o interface
Difficulty 3/5 1-2 days Newbie friendliness 65/100
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·