Python: CodeTokenizer splits quoted named-arg values on spaces (NamedArgBlockSyntaxError)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start with CodeTokenizer in semantic_kernel/template_engine/code_tokenizer and run the provided tokenize reproduction for both single- and double-quoted values. Done means each quoted named-argument value containing spaces remains one token and the example templates tokenize without NamedArgBlockSyntaxError.
Written by the indexing model from the issue text.
Description
Description
CodeTokenizer splits named-argument quoted values at spaces, so a template like:
{{func key='hello world'}}
fails with NamedArgBlockSyntaxError: A NamedArgBlock starts with a name... — the token for the argument is cut to key='hello and parsing breaks. Any quoted named-arg value containing a space breaks the whole template.
Reproduction
from semantic_kernel.template_engine.code_tokenizer import CodeTokenizer
CodeTokenizer().tokenize("{{func key='hello world'}}")
# NamedArgBlockSyntaxError
Same with double quotes.
Expected behavior
quoted values are single tokens, spaces included, matching how the value grammar is documented (ValBlock is text surrounded by quotes).
Environment
semantic-kernel python main, Python 3.12
- Dominant language
- C#
- Stars
- 28.6k
- Forks
- 4.8k
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 18
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/semantic-kernel
-
python triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
microsoft/semantic-kernel#14443 ·
-
Python: @kernel_function mutates shared Annotated metadata dicts (description lost after first use) Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
microsoft/semantic-kernel#14441 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
microsoft/semantic-kernel#14359 · 3 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
microsoft/semantic-kernel#14347 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
microsoft/semantic-kernel#14311 · 2 comments ·
All issues in microsoft/semantic-kernel
Similar issues
-
type/automation type/tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
t/bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
ci-failure-cause test-failure
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
area:auth FE mvp P3
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
klasolsson81/jobbliggaren#1788 ·