Linalg autograd blog post: Typo/error in NumPy gradient computation for linalg.inv

Open Beginner friendly
#2,135 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
numpy, python
Domain
documentation

Research direction

Open the torch-linalg-autograd blog post and locate the NumPy inv_backward example. Replace the shown function with the corrected computation, then verify it using the asymmetric matrix from the issue so the gradient no longer relies on a symmetric result.

Written by the indexing model from the issue text.

Description

I noticed that the Autograd support example here:
https://pytorch.org/blog/torch-linalg-autograd/
has an error, because NumPy result.transpose(-2,-1) behaves differently from PyTorch and does not transpose as expected. While the example matrix happens to work (the resulting mat is symmetric), the snippet would break for example with:

a =  np.array([[0.0, -1.0, 0.0],[1.0, 0.0, 0.0], [0.0, 0.0, 1.0]])

The error can be fixed by replacing the inv_backward function with

def inv_backward(result, grad):
    return -(result.T @ (grad @ result.T))
Dominant language
HTML
Stars
285
Forks
317
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pytorch/pytorch.github.io

All issues in pytorch/pytorch.github.io

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.