lexer freezes on this Python code
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- javascript, python
- Domain
- compilers
Research direction
Start by reproducing the freeze with lexer("print(''m')") and inspect the lexer function, especially the matchQuotation loop shown in the issue. Trace how escaped quotes are tokenized and ensure this input terminates without advancing past the end of input; verify that the lexer no longer hangs.
Written by the indexing model from the issue text.
Description
Steps to reproduce:
- Try to run
lexer("print('\'m')") - Notice that it freezes.
I traced the freeze to an infinite do-while loop in the lexer.
In the following code, input is set to the code "print('\'m')" but current variable reaches into the billions,
currentChar === undefined, and
validator.test(currentChar) is returning false when currentChar is undefined.
/**
* 过滤（提取） 引号中的内容
*/
// eslint-disable-next-line
const matchQuotation = (currentChar, validator, TokenType) => {
do {
if (currentChar === '\n') {
line++;
}
currentChar = input[++current];
} while (!validator.test(currentChar));
++current;
};
I wonder if this has anything to do with the backslashed quotation mark. Maybe the grammar doesn't define STRING properly for escaped quotes.
I edited this issue because I first thought the bug was reproducible by parsing but it actually requires running the lexer function on the code.
- Dominant language
- JavaScript
- Stars
- 25
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DTStack/dt-python-parser
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
DTStack/dt-python-parser#28 ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
DTStack/dt-python-parser#27 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
DTStack/dt-python-parser#26 ·
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
DTStack/dt-python-parser#15 · 3 comments ·
-
支持最新版Python3语法 Open
Difficulty 5/5 Over a week Newbie friendliness 20/100
DTStack/dt-python-parser#11 ·
All issues in DTStack/dt-python-parser
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
palladius/rails8-app-on-gcp#145 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
dotenvx/dotenv-vscode#139 ·
-
test-change-proposal
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
web-platform-tests/interop#1455 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
corsairdev/corsair#1764 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100