arogozhnikov/einops

Missing einops.layers.....Repeat

Open

#185 opened on Apr 26, 2022

View on GitHub
 (5 comments) (3 reactions) (0 assignees)Python (309 forks)batch import
feature suggestiongood first issue

Repository metrics

Stars
 (7,080 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

I tried to use repeat for in torch and needed a layer, but strangely it was not there. I know, that I could use einops.layers.torch.Reduce('...', reduction='repeat', ...), but that is confusing to read.

What do you think about adding einops.layers.....Repeat functions to einops?

Here is a toy example, where the last line fails because the function counterpart is missing and the first line is difficult to read:

import einops

t = torch.randn((2, 3))
print(einops.layers.torch.Reduce('a b -> a b c', reduction='repeat', c=4)(t).shape)  # torch.Size([2, 3, 4])
print(einops.repeat(t, 'a b -> a b c', c=4).shape)  # torch.Size([2, 3, 4])
print(einops.layers.torch.Repeat('a b -> a b c', reduction='repeat', c=4)(t).shape)  # AttributeError: module 'einops.layers.torch' has no attribute 'Repeat'

  1. Try to collect use-cases

Since einops.repeat exists, I think the same use cases would be valid for a layer. I have one case, where I want to use it in pytorch.

  1. Implementation. (optional) Implementing a sketch of your proposal in a code allows detecting possible conflicts and realize possible caveats.

Something like the following in each layers backend file

class Repeat(Reduce):
    def __init__(self, pattern, **axes_lengths):
        super().__init__(pattern, 'repeat', **axes_lengths)
  1. Integrity - does it interplay well with existing operations and notation in einops?

I would say yes, it is the counterpart of the function einops.repeat.

  1. Readability. This is harder to check, but give it a try. A simple but usable test is to write an exercise sheet with several examples of your extension explained, for others meaning of operation should be guessed. Send this test to a couple of your friends to collect the feedback and see how your notion will be understood. This should also help to improve your ideas to make them more digestible.

I think this is obvious, currently I use einops.layers.torch.Reduce('...', reduction='repeat', ...) and that is confusing.

Contributor guide