TextSlice implementation breaks codebases based on 1.2.2
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 64/100
Research direction
Start by locating the TextSlice implementation involved in PR #1452 and inspect how lexer_state.text.text is accessed. Reproduce the TypeError with the slicing pattern described in this issue, then verify that TextSlice supports the required slice access without breaking existing behavior. The escapy example and linked issue #1211 provide usage context.
Written by the indexing model from the issue text.
Description
Hi,
PR #1452 broke the code that uses the library's internal API.
I am developing escapy and I am using an implementation based on the code snippet provided here https://github.com/lark-parser/lark/issues/1211#issuecomment-1304367740.
I rely on an InteractiveParser to intercept tokens from the lexer coming from a file and, by bypassing the limitations of context-free grammar, generate tokens that could not be handled otherwise (context-sensitive) (see eventually https://github.com/ysard/escapy/blob/3a8939c7fd66b1210de9ab140c4a46273230307a/escapy/grammar.py#L630).
The lexer’s internal state is heavily used, particularly the internal text attribute.
The changes in #1452 add a level of indirection to this attribute, requiring a modification to the lexer_state.text.text code.
The old code therefore generates TypeError: ‘TextSlice’ object is not subscriptable errors with version 1.3.1 (see https://github.com/ysard/escapy/issues/3).
Perhaps my implementation isn’t the one you’d recommend for my use case, but it’s the only one I’ve found...
If you would add a Python magic method (see implementation below) to the TextSlice object, you could:
- lighten your codebase
- avoid breaking existing code that relies on the old implementation (no API breakage = no need to release a major revision of Lark).
Something like this works perfectly:
class TextSlice(Generic[AnyStr]):
...
def __getitem__(self, key):
if isinstance(key, slice):
return self.text[key]
else:
raise TypeError("Invalid argument type.")
- Dominant language
- Python
- Stars
- 6k
- Forks
- 523
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lark-parser/lark
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
lark-parser/lark#1608 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
lark-parser/lark#1526 · 3 comments ·
-
docs good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
lark-parser/lark#1494 · 2 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
lark-parser/lark#1445 · 1 reaction ·
-
Zero-width regex terminal rejected even when the surrounding rule can never match zero characters Open
Difficulty 4/5 3-5 days Newbie friendliness 68/100
lark-parser/lark#1640 · 1 comment ·
All issues in lark-parser/lark
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100