Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Jinja templates see string cells that look like JSON scalars as numbers, booleans and None

Aperta
  • Qualcosa non va?
#1,088 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
pandas, python
Ambito
backend

Direzione di ricerca

Start in deserialize_json_values in packages/data-designer-engine/src/data_designer/engine/processing/utils.py (string cases around lines 87 and 101), then read its callers in expression.py:62 and llm_completion.py:81 and the skip-condition evaluation. The bug report includes a runnable repro script; run it to see the wrong rendering, and note issue #994/#1059 discussed the same conversion for embeddings. Done means a maintainer has picked one of the proposed directions (keep scalars as text, per-column opt-in, or document) and the repro renders cell text unchanged without breaking configs like {{ n + 1 }}. A triage decision is still needed, and the reporter awaits that direction before sending a PR.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Priority Level

Medium (Annoying but has workaround)

Describe the bug

Before a template is rendered, every string value in the record goes through json.loads. The comment in llm_completion.py says this is so {{ prev_column.key }} works when an earlier column stored a JSON object, but it also fires on strings that are valid JSON scalars. So a cell with the text 1.10 reaches the template as the float 1.1, true becomes True, null becomes None and 1e3 becomes 1000.0. The rendered text no longer matches what is in the column.

The conversion happens in deserialize_json_values in packages/data-designer-engine/src/data_designer/engine/processing/utils.py (lines 87 and 101 handle strings inside a list or dict, which is the record case). It is called on the record by the expression generator (expression.py:62), the LLM completion generators when building the prompt (llm_completion.py:81), the image generator, skip.when evaluation and the schema transform processor. Calling evaluate_skip_when("{{ flag == 'true' }}", {"flag": "true"}) directly returns False, so a skip condition written against the text true never matches. I did not run the image generator or the schema transform.

Strings that are not valid JSON are left alone, so 007 and yes come through as written. The result depends on the value, and one column can render some rows correctly and others not.

Steps/Code to reproduce bug

On main at 207eda6d, Python 3.14.4, no model or API key needed.

import pandas as pd

import data_designer.config as dd
from data_designer.interface import DataDesigner

seed = pd.DataFrame(
    {
        "version": ["1.10", "2.0", "1e3", "007"],
        "flag": ["true", "false", "null", "yes"],
    }
)

builder = dd.DataDesignerConfigBuilder()
builder.with_seed_dataset(dd.DataFrameSeedSource(df=seed))
builder.add_column(dd.ExpressionColumnConfig(name="label", expr="v{{ version }}"))
builder.add_column(dd.ExpressionColumnConfig(name="flag_copy", expr="{{ flag }}"))
builder.add_column(dd.ExpressionColumnConfig(name="is_true", expr="{{ flag == 'true' }}"))

preview = DataDesigner().preview(builder, num_records=4)
print(preview.dataset[["version", "label", "flag", "flag_copy", "is_true"]].to_string())

Output

  version    label   flag flag_copy is_true
0    1.10     v1.1   true      True   False
1     2.0     v2.0  false     False   False
2     1e3  v1000.0   null      None   False
3     007     v007    yes       yes   False

It is not specific to DataFrame seeds. A category sampler with values=["1.10", "true"] feeding v{{ api }} rendered v1.1 and vTrue in the same kind of preview, and a CSV seed with a mixed column (1.10, 1.9, beta) kept the column as strings and still rendered 1.10 as 1.1. I also pointed an LLMTextColumnConfig at a local OpenAI-compatible stub that echoes the user message back. With the prompt Write release notes for v{{ version }}, beta={{ flag }} and the seed row 1.10 / true, the model received Write release notes for v1.1, beta=True.

Expected behavior

I would expect a string cell to render as the text that is in the cell, so v{{ version }} gives v1.10, {{ flag }} gives true and {{ flag == 'true' }} is true.

That said, some configs probably depend on the current behavior. With a seed column of strings "5" and "6", {{ n + 1 }} renders 6 and 7 on main today because "5" is turned into an int first. I tried a version of the function that only replaces nested strings when they parse to a JSON object or array and keeps scalar strings as text. The repro above then prints v1.10, true and True as expected, but the {{ n + 1 }} preview fails with Expression column 'n_plus_1' produced no valid rows, so that would need | int or | float in the template.

Which direction do you prefer? I see a few options. Keep scalar strings as text and call the change out in release notes. Make it opt-in or opt-out per column. Or keep the current behavior and document it. Note that | string does not help here. v{{ version | string }} still renders v1.1.

Agent Diagnostic / Prior Investigation

Searching open and closed issues and PRs for deserialize_json_values, numeric strings and string-to-bool conversion in templates found nothing that reports this. The closest items are #994 and #1059, where the embedding column hit the same conversion and was fixed locally in embedding.py, leaving deserialize_json_values as it is. In #994 the review asked that strings like "123", "true" and "null" keep their original text for embeddings, which is the same question for templates. #232 / #233 stopped deserializing the LLM text response itself, but the values are still converted when a later template reads them. No open PR touches processing/utils.py, expression.py or skip_evaluator.py. I did not find the conversion described in the docs.

Additional context

The only workaround I see today is to keep the stored values from being valid JSON, for example storing v1.10 instead of 1.10. That is hard when the values come from a seed file you do not control or from an LLM. I can send a PR once this is triaged and a direction is picked.

Checklist
  • I reproduced this issue or provided a minimal example
  • I searched the docs/issues myself, or had my agent do so
  • If I used an agent, I included its diagnostics above
Lingua principale
Python
Stelle
2.4k
Fork
241
Merge medio
1g 19m
PR unite (30g)
84

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA-NeMo/DataDesigner

Tutte le issue di NVIDIA-NeMo/DataDesigner

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.