easygood first issuehelp wanted
Metriche repository
- Star
- (10 star)
- Metriche merge PR
- (Metriche PR in attesa)
Descrizione
Currently, our concatenation operation on dataframes doesn't remember which rows came from which file. It would be nice to keep this information. This is easy to do in by just modifying load_dsts. However, the design of the output is not obvious. We could
- Simply add a column with the file name. This is the simplest solution, but it adds a lot of noise and unnecessary data to the dst. Large dsts may become to heavy to be held in memory.
- Add a smart file indexer that maybe adds just the index of the file being loaded. This relies heavily on the file being read to be in the standard filename format and on this format not changing. Might be a pain in the future.
- Make the output more complicated. Add only
file_indexcolumn to the main dataframe, but return also a second dataframe that associatesfile_indexto the actual file name. This seems the most robust solution, but also the most annoying for the user, maybe. - Do something else
People interested in this should comment what their preference is or propose other schemes.