arogozhnikov/einops

Missing einops.layers.....Repeat

Open

#185 aberto em 26 de abr. de 2022

Ver no GitHub
 (5 comments) (3 reactions) (0 assignees)Python (309 forks)batch import
feature suggestiongood first issue

Métricas do repositório

Stars
 (7.080 stars)
Métricas de merge de PR
 (Nenhuma PRs mesclada em 30d)

Description

I tried to use repeat for in torch and needed a layer, but strangely it was not there. I know, that I could use einops.layers.torch.Reduce('...', reduction='repeat', ...), but that is confusing to read.

What do you think about adding einops.layers.....Repeat functions to einops?

Here is a toy example, where the last line fails because the function counterpart is missing and the first line is difficult to read:

import einops

t = torch.randn((2, 3))
print(einops.layers.torch.Reduce('a b -> a b c', reduction='repeat', c=4)(t).shape)  # torch.Size([2, 3, 4])
print(einops.repeat(t, 'a b -> a b c', c=4).shape)  # torch.Size([2, 3, 4])
print(einops.layers.torch.Repeat('a b -> a b c', reduction='repeat', c=4)(t).shape)  # AttributeError: module 'einops.layers.torch' has no attribute 'Repeat'

  1. Try to collect use-cases

Since einops.repeat exists, I think the same use cases would be valid for a layer. I have one case, where I want to use it in pytorch.

  1. Implementation. (optional) Implementing a sketch of your proposal in a code allows detecting possible conflicts and realize possible caveats.

Something like the following in each layers backend file

class Repeat(Reduce):
    def __init__(self, pattern, **axes_lengths):
        super().__init__(pattern, 'repeat', **axes_lengths)
  1. Integrity - does it interplay well with existing operations and notation in einops?

I would say yes, it is the counterpart of the function einops.repeat.

  1. Readability. This is harder to check, but give it a try. A simple but usable test is to write an exercise sheet with several examples of your extension explained, for others meaning of operation should be guessed. Send this test to a couple of your friends to collect the feedback and see how your notion will be understood. This should also help to improve your ideas to make them more digestible.

I think this is obvious, currently I use einops.layers.torch.Reduce('...', reduction='repeat', ...) and that is confusing.

Guia do colaborador