Dynamic dispatching in optimizers
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Refactor
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- julia
- Domain
- machine-learning
Research direction
Start by reading src/optimizers/sgd.jl and src/optimizers/adam.jl, especially each update method and the get_learning_rate call. Confirm how optimizer options and lr_scheduler are typed, then determine whether dispatch is dynamic and whether a change is warranted. Done requires a documented conclusion or an agreed implementation scope, but the issue does not name tests or an acceptance criterion.
Written by the indexing model from the issue text.
Description
I could be wrong in this, but looking at the code for sgd and adam it looks like there is dynamic function dispatch going on because the types for optimizer options are not known at compile time.
For example for sgd in update:
lr = get_learning_rate(self.opts.lr_scheduler, self.state)
If I understand Julia correctly, then lr_scheduler is a boxed value in an Any block and thus the appropriate get_learning_rate variant needs to be looked up at runtime, which can have performance implications.
What do you think?
- Dominant language
- No language data
- Stars
- 369
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from dmlc/MXNet.jl
-
Import fail MXNET.jlOpen
Difficulty 3/5 1-2 days Newbie friendliness 28/100
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Similar issues
-
Questions crashes on a real SQuAD2 no-answer predictionPossibly taken @HuaTNA claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
huggingface/sentence-transformers#4186 ·
Maintainers usually reply within 5 days
-
bot-triaged module: correctness (silent) module: mps module: NaNs and Infs triage review
Difficulty 2/5 Half a day Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/structured-data-models#1156 ·
Maintainers usually reply within 1 day
-
`_AdverserialFairness` base class should not set the classes_ attribute for regressorsPossibly taken @kocaemre claimed this today. Openbug python
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day