rows_from_file() ignores ignore_extras= and extras_key= for anything that isn't Format.CSV

Ouverte Adaptée aux débutants
  • Quelque chose ne va pas ?
#892 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 3 jours

@feiiiiii5 y travaille déjà.

Depuis le 30/9/2026.

  • #893 par @feiiiiii5 — ouverte

Évaluation

Difficulté
2/5
Temps estimé
1-3 heures
Accessibilité débutants
88/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
python
Domaine
tooling

Piste de recherche

Commencez dans sqlite_utils/utils.py, au niveau de rows_from_file(), et examinez tests/test_rows_from_file.py, en particulier le test CSV existant à la ligne 44. Exécutez les tests de rows_from_file, puis couvrez les chemins TSV et détectés automatiquement explicitement mentionnés, tout en préservant le comportement RowError par défaut. C'est terminé lorsque ignore_extras et extras_key fonctionnent de manière cohérente pour ces chemins sans modifier la valeur par défaut documentée.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

rows_from_file() ignores ignore_extras= and extras_key= for anything that isn't explicitly format=Format.CSV.

The docstring says:

If a CSV or TSV file includes rows with more fields than are declared in the header a sqlite_utils.utils.RowError exception will be raised when you loop over the generator.

You can instead ignore the extra data by passing ignore_extras=True.

Or pass extras_key="rest" to put those additional values in a list in a key called rest.

In practice those two options only work for format=Format.CSV. For TSV — which the docstring names explicitly — and for the auto-detected path (format omitted) they are silently discarded, and the caller's own explicit request is contradicted by a RowError.

Both branches that recurse into the CSV branch call rows_from_file() without forwarding the two arguments, so the inner call uses its own defaults (ignore_extras=False, extras_key=None) and raises before the outer _extra_key_strategy() wrapper — which is dead code — ever sees the row:

>>> from io import BytesIO
>>> from sqlite_utils.utils import rows_from_file, Format
>>> rows, _ = rows_from_file(
...     BytesIO(b"id\tname\r\n1\tCleo\toops"),
...     format=Format.TSV,
...     ignore_extras=True,
... )
>>> list(rows)
Traceback (most recent call last):
  ...
sqlite_utils.utils.RowError: Row {'id': '1', 'name': 'Cleo'} contained these extra values: ['oops']

The same happens with extras_key="_rest":

>>> rows, _ = rows_from_file(
...     BytesIO(b"id\tname\r\n1\tCleo\toops"),
...     format=Format.TSV,
...     extras_key="_rest",
... )
>>> list(rows)
Traceback (most recent call last):
  ...
sqlite_utils.utils.RowError: Row {'id': '1', 'name': 'Cleo'} contained these extra values: ['oops']

With no options passed the behaviour is correct and should not change — RowError is still raised, as documented.

git log -S"_extra_key_strategy" points at d379f43 ("rows_from_file(... ignore_extras: bool, restkey: str), refs #440"), which introduced the strategy and both branches in one commit. The recursion has never forwarded the arguments, so this looks like an oversight in the original commit rather than a decision.

The existing test hardcoded format=Format.CSV (tests/test_rows_from_file.py:44), which is why it was never caught.

Proposed fix

Forward ignore_extras= and extras_key= to the inner call in both branches, and drop the now-redundant outer _extra_key_strategy wrapper so the strategy is applied once, on the shared CSV path that already implements it.

Note on scope

I would fix the auto-detect branch in the same commit, since it is the same root cause (same dropped-argument recursion, same dead wrapper) and would otherwise be a one-line follow-up. It interacts with #891: once the sniffer picks the right delimiter for a ragged file, the auto-detected path will honour ignore_extras too.

I did not add an auto-detect regression test, because csv.Sniffer returns the wrong delimiter on a small ragged input and pinning today's sniffing behaviour would bake in the bug #891 is about. The TSV test passes an explicit format, so it does not depend on the sniffer.

Langage dominant
Python
Étoiles
2.2k
Forks
215
Aucune PR mergée en 30 j

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de simonw/sqlite-utils

Toutes les issues de simonw/sqlite-utils

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.