patchedast: visitors that omit sub-nodes leave unconsumed text that later token searches mis-match

Offen
  • Stimmt etwas nicht?
#912 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 7 Tagen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
46/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
python
Bereich
devtools

Rechercherichtung

Start in rope/base/ast.py and trace _PatchingASTWalker plus the visitors named in the issue, including _arguments, _arg, _handle_function_def_node, _JoinedStr, _ClassDef and _TryFinally. Reproduce the listed cases with patchedast.get_patched_ast(src, sorted_children=True), then add a regression check that relevant AST nodes and string literals receive regions covering their AST spans. Done means the cases no longer raise or leave source text without a region.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

_PatchingASTWalker finds each token with a forward source.index, so any source text a visitor does not
list is skipped as whitespace; a (, ), = or quote inside it is then matched as a later token. The
walk raises MismatchedTokenError / AttributeError: 'NoneType' object has no attribute 'group', or it
silently produces wrong regions, which Scope.get_region uses for scope lookup. On rope 1.15.0, each
repro below makes patchedast.get_patched_ast(src, sorted_children=True) either raise or finish with part
of the source left without a region:

  1. posonly / kw-only args and kw_defaults (_arguments): def f(*, m: str = "(host)"): pass
  2. parameter annotations (_arg): def f(p: A[F(d="t")] = None): pass
  3. return annotations (_handle_function_def_node): def f() -> A[F(d="(")]: pass
  4. *args / **kwargs annotations: def f(*a: F("("), **k: F(")")): pass
  5. f-string implicitly concatenated with a later literal (_JoinedStr stops at the f-string's quote):
    c = g(f"{d}" "(")
  6. class keywords (_ClassDef): class A(B, metaclass=M, x=")"): pass
  7. else of try/except/else/finally (_TryFinally): an else: body holding x = "("
  8. rb/Rb prefixes (string pattern [uUbB]?[rR]?): x = rb"("; the region starts after r
  9. 3.14 t-strings: TemplateStr/Interpolation are unknown nodes: f(t"{a}(", ")")
  10. 3.12+ nested f-strings that reuse the outer quote: x = f"{f"{a}("}" ")" (the string regex stops at
    the inner quote)

Suggested regression test: after walking any module, every AST node except context/operator nodes,
withitem and f-string internals has a region, and every string literal's region covers its
ast span. With that check, CPython's stdlib and a 1,200-file project pass once the visitors above are
fixed. We carry these patches in living-architecture's rope_patches.py and will drop each when it
lands upstream.

Vorherrschende Sprache
Python
Sterne
2.2k
Forks
197
Ø Merge
2 T. 13 Std.
Gemergte PRs (30 T.)
5

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus python-rope/rope

Alle Issues in python-rope/rope

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.