register_hook

Open
#1,483 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp, python
Domain
machine-learning

Research direction

Start with the Tensor autograd entry points shown in the issue: states.register_hook, loss.register_hook, and loss.backward, along with the model.represent rollout call. Compare TorchSharp's available gradient APIs with the Python example, then define and verify a supported way to scale the intermediate states gradient without changing the loss scaling behavior.

Written by the indexing model from the issue text.

Description

question

I cannot find a method to register a hook. Is there another way this can be accomplished?

for step_i in range(unroll_steps):
   states = model.represent(states)
   states.register_hook(lambda grad: grad * 0.5)
   #Get the loss...

loss.register_hook(1.0/unroll_steps)
optimizer.zero_grad()
loss.backward()

The above is an example in python.

I believe I can do the loss gradient like this:

List<Tensor> grads = [torch.tensor(1.0f / unroll_steps)];
loss.backward(grads);

But I am still unsure how to scale the gradient of the 'states' tensor. Any help would be appreciated.

Just for background context. This is an implementation of MuZero. In their code, the scale the future latent states non-linearly. This gradient happens in the rollout loop prior to the loss gradient which is scaled linearly.

  • OS: Windows
  • Package Type: torchsharp-cuda-windows
  • Version: 0.105.0
Dominant language
C#
Stars
1.9k
Forks
228
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/TorchSharp

All issues in dotnet/TorchSharp

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.