LLM Quantization Cannot Load `text_dataset` (Default and Custom Values Fail)

Aperta

#150 aperta il 27/7/2026 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@mihai-chiorean ci sta già lavorando.

Dal 19/10/2026.

#255 di @mihai-chiorean — aperta

Valutazione

Difficoltà: 3/5
Tempo stimato: 1-2 giorni
Idoneità per principianti: 65/100

Tipo di issue: Bug
Chiarezza: Abbastanza chiara
Stato di attività: Tranquilla
Stack tecnologico: python
Ambito: machine-learning

Direzione di ricerca

Inizia da tensorrt_edgellm/quantization/datasets/builtin.py e datasets/init.py, quindi segui la chiamata da quantization/quantize.py:805. Esegui nuovamente il comando documentato tensorrt-edgellm-quantize sia con cnn_dailymail sia con abisee/cnn_dailymail. Il lavoro è completato quando i valori del dataset documentati vengono caricati correttamente per la calibrazione del testo senza gli errori di repository o di dataset registrato mostrati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Describe the bug

I cannot reproduce the quantization step from the Quick Start Guide because the text_dataset argument consistently fails.

When set to cnn_dailymail, it returns:

Repository id must be 'namespace/name', got 'cnn_dailymail'.

When set to abisee/cnn_dailymail, it returns:

Registered text datasets: cnn_dailymail, wikitext.

Steps/Code to reproduce bug

Installation method:
As described in TensorRT Edge-LLM on Jetson, I used an NVIDIA Jetson Thor as both the host and target device. I installed TensorRT Edge-LLM v0.9.1 by following the Installation Guide. The installation completed successfully without any issues.

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jul_16_07:31:19_PM_PDT_2025
Cuda compilation tools, release 13.0, V13.0.48
Build cuda_13.0.r13.0/compiler.36260728_0

$ nvidia-smi
Mon Jul 27 08:52:49 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00                 Driver Version: 580.00         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA Thor                    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   36C  N/A               3W /  N/A  | Not Supported          |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A           15507      G   /usr/lib/xorg/Xorg                       92MiB |
|    0   N/A  N/A           15707      G   /usr/bin/gnome-shell                     55MiB |
+-----------------------------------------------------------------------------------------+

$ python3 --version
Python 3.12.3

$ cat /etc/nv_tegra_release
# R38 (release), REVISION: 4.0, GCID: 43443517, BOARD: generic, EABI: aarch64, DATE: Wed Dec 31 00:15:19 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
INSTALL_TYPE=

Export command used:

When text_dataset is set to cnn_dailymail (also default value):

$ tensorrt-edgellm-quantize llm --model_dir Qwen/Qwen3-0.6B --output_dir Qwen3-0.6B-My-fp8-fp16 --quantization fp8 --text_dataset cnn_dailymail

/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/modelopt/torch/__init__.py:53: UserWarning: transformers>=5.0 support is experimental. Unified Hugging Face checkpoint export for quantized checkpoints may not work for some models yet.
_warnings.warn(
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 311/311 [00:00<00:00, 3993.24it/s]
Text calibration dataset: cnn_dailymail
Traceback (most recent call last):
File "/workspace/TensorRT-Edge-LLM/host_venv/bin/tensorrt-edgellm-quantize", line 8, in <module>
    sys.exit(main())
            ^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/quantize.py", line 171, in main
    quantize_and_export(
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/quantize.py", line 805, in quantize_and_export
    loader = _text_calib_dataloader(tokenizer,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/quantize.py", line 62, in _text_calib_dataloader
    texts = list(islice(text_dataset(), num_samples))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/builtin.py", line 27, in cnn_dailymail
    ds = load_hf_split("cnn_dailymail",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/__init__.py", line 238, in load_hf_split
    return load_dataset(hf_id, config, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1688, in load_dataset
    builder_instance = load_dataset_builder(
                    ^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1315, in load_dataset_builder
    dataset_module = dataset_module_factory(
                    ^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1207, in dataset_module_factory
    raise e1 from None
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1182, in dataset_module_factory
    ).get_module()
    ^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 598, in get_module
    standalone_yaml_path = cached_path(
                        ^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/utils/file_utils.py", line 180, in cached_path
    ).resolve_path(url_or_filename)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/hf_file_system.py", line 305, in resolve_path
    parsed = parse_hf_uri(f"{constants.HF_PROTOCOL}{path}")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/utils/_hf_uris.py", line 319, in parse_hf_uri
    return _parse_repo_body(location, type_, raw=raw)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/utils/_hf_uris.py", line 617, in _parse_repo_body
    raise HfUriError(uri=raw, msg=f"Repository id must be 'namespace/name', got '{repo_id}'.")
huggingface_hub.errors.HfUriError: Invalid HF URI 'hf://datasets/cnn_dailymail@96df5e686bee6baa90b8bee7c28b81fa3fa6223d/.huggingface.yaml'. Repository id must be 'namespace/name', got 'cnn_dailymail'.

When text_dataset is set to abisee/cnn_dailymail:

$ tensorrt-edgellm-quantize llm --model_dir Qwen/Qwen3-0.6B --output_dir Qwen3-0.6B-My-fp8-fp16 --quantization fp8 --text_dataset abisee/cnn_dailymail

/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/modelopt/torch/__init__.py:53: UserWarning: transformers>=5.0 support is experimental. Unified Hugging Face checkpoint export for quantized checkpoints may not work for some models yet.
_warnings.warn(
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 311/311 [00:00<00:00, 3993.24it/s]
Text calibration dataset: cnn_dailymail
Traceback (most recent call last):
File "/workspace/TensorRT-Edge-LLM/host_venv/bin/tensorrt-edgellm-quantize", line 8, in <module>
    sys.exit(main())
            ^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/quantize.py", line 171, in main
    quantize_and_export(
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/quantize.py", line 805, in quantize_and_export
    loader = _text_calib_dataloader(tokenizer,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/quantize.py", line 62, in _text_calib_dataloader
    texts = list(islice(text_dataset(), num_samples))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/builtin.py", line 27, in cnn_dailymail
    ds = load_hf_split("cnn_dailymail",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/__init__.py", line 238, in load_hf_split
    return load_dataset(hf_id, config, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1688, in load_dataset
    builder_instance = load_dataset_builder(
                    ^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1315, in load_dataset_builder
    dataset_module = dataset_module_factory(
                    ^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1207, in dataset_module_factory
    raise e1 from None
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 1182, in dataset_module_factory
    ).get_module()
    ^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/load.py", line 598, in get_module
    standalone_yaml_path = cached_path(
                        ^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/datasets/utils/file_utils.py", line 180, in cached_path
    ).resolve_path(url_or_filename)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/hf_file_system.py", line 305, in resolve_path
    parsed = parse_hf_uri(f"{constants.HF_PROTOCOL}{path}")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/utils/_hf_uris.py", line 319, in parse_hf_uri
    return _parse_repo_body(location, type_, raw=raw)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/huggingface_hub/utils/_hf_uris.py", line 617, in _parse_repo_body
    raise HfUriError(uri=raw, msg=f"Repository id must be 'namespace/name', got '{repo_id}'.")
huggingface_hub.errors.HfUriError: Invalid HF URI 'hf://datasets/cnn_dailymail@96df5e686bee6baa90b8bee7c28b81fa3fa6223d/.huggingface.yaml'. Repository id must be 'namespace/name', got 'cnn_dailymail'.
(host_venv) root@d1cdc56c1c93:/workspace/tensorrt-edgellm-workspace/LLM# tensorrt-edgellm-quantize llm --model_dir Qwen/Qwen3-0.6B --output_dir Qwen3-0.6B-My-fp8-fp16 --quantization fp8 --text_dataset abisee/cnn_dailymail
/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/modelopt/torch/__init__.py:53: UserWarning: transformers>=5.0 support is experimental. Unified Hugging Face checkpoint export for quantized checkpoints may not work for some models yet.
_warnings.warn(
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 311/311 [00:00<00:00, 3688.66it/s]
Traceback (most recent call last):
File "/workspace/TensorRT-Edge-LLM/host_venv/bin/tensorrt-edgellm-quantize", line 8, in <module>
    sys.exit(main())
            ^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/quantize.py", line 171, in main
    quantize_and_export(
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/quantize.py", line 802, in quantize_and_export
    text_ds = resolve_dataset(text_dataset, "text")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/__init__.py", line 152, in resolve_dataset
    return get_dataset(modality, value)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/quantization/datasets/__init__.py", line 133, in get_dataset
    raise UnknownCalibrationDataset(modality, name,
tensorrt_edgellm.quantization.datasets.UnknownCalibrationDataset: Unknown text calibration dataset 'abisee/cnn_dailymail'. Registered text datasets: cnn_dailymail, wikitext. To calibrate on a custom dataset, write a text generator and register it with @register('text', ...). See the customization guide (docs/source/developer_guide/customization/calibration-datasets.md) and copy the template at tensorrt_edgellm/quantization/datasets/custom_dataset_template.py.

System information (Nvidia Jetson Thor)

Container used: nvcr.io/nvidia/pytorch:25.12-py3
OS: Ubuntu 24.04.3 LTS
CPU architecture: aarch64
GPU name: Nvidia Jetson T5000
GPU memory size: 128 GB
Number of GPUs: 1
Library versions:
- Python: 3.12.3
- TensorRT Edge-LLM version or commit hash: 0.9.1
- CUDA: 13.1
- PyTorch: 2.12.0+cu130
- Transformers: 5.9.0
- ModelOpt: 0.44.0
- ONNX: 1.19.0

Lingua principale: Python
Stelle: 613
Fork: 146
Merge medio: 2g 19h
PR unite (30g): 7

Preparare l'ambiente

Nessun Dockerfile né file Docker Compose
Ha un modello di pull request
Leggi la guida per i contributori

Come iniziare

Leggi tutta la issue e poi la guida ai contributi del progetto.
Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
Fai un fork del repository e lavora su un branch.
Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/TensorRT-Edge-LLM

Windows source build fails with C2001 in xgrammar's tokenizer_info.cc when the system code page is 932
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100

NVIDIA/TensorRT-Edge-LLM#277 · Aperta il 7/11/2026

I maintainer di solito rispondono entro 1 giorno
GPTQ MoE export ignores act-order group indices
Forse già presa @0z5a l’ha presa 3 giorni fa. Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100

NVIDIA/TensorRT-Edge-LLM#272 · Aperta il 4/11/2026

I maintainer di solito rispondono entro 1 giorno
ENABLE_CUTE_DSL=OFF does not compile: runPadding stub does not match its declaration
Forse di nuovo libera @akhil9tiet l’ha presa 7 giorni fa e non c’è nessuna pull request aperta. Aperta
- Qualcosa non va?
bug

Difficoltà 1/5 Meno di un'ora Idoneità per principianti 82/100

NVIDIA/TensorRT-Edge-LLM#268 · 2 commenti · 1 reazione · Aperta il 1/11/2026

I maintainer di solito rispondono entro 1 giorno
docs: Jetson build prerequisites omit the NVRTC headers (cuda-nvrtc-dev) and the TensorRT plugin headers
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100

NVIDIA/TensorRT-Edge-LLM#244 · Aperta il 10/10/2026

I maintainer di solito rispondono entro 1 giorno
llm_inference ignores temperature/top_p/top_k from the input JSON its own help tells you to set them in
Forse già presa @filipemartinsubrobotics l’ha presa 56 giorni fa. Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100

NVIDIA/TensorRT-Edge-LLM#211 · 2 commenti · Aperta il 12/9/2026

I maintainer di solito rispondono entro 1 giorno

Tutte le issue di NVIDIA/TensorRT-Edge-LLM

Issue simili

[yugabyted] xcluster set_up and add_to_replication fail for database names containing - . or (
Aperta
- Qualcosa non va?
kind/bug priority/medium qa_automation

Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100

yugabyte/yugabyte-db#37324 · Aperta il 8/11/2026

I maintainer di solito rispondono entro 1 giorno
get_chain_tips's docstring points at the wrong lines of Core's getchaintips
Aperta
- Qualcosa non va?
good first issue

Difficoltà 1/5 Meno di un'ora Idoneità per principianti 87/100

btclib-org/btclib-node#2260 · Aperta il 8/11/2026

I maintainer di solito rispondono entro 1 giorno
ChatBedrockConverse changes the keys of JSON tool results to camelCase (`order_id` becomes `orderId`)
Forse già presa @geyazheng l’ha presa oggi. Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100

langchain-ai/langchain-aws#1469 · Aperta il 8/11/2026

I maintainer di solito rispondono entro 1 giorno
GCToggler leaves collection enabled if the body re-enables it
Forse già presa @Gambit-Checkmate l’ha presa oggi. Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100

mahmoud/boltons#745 · Aperta il 8/11/2026

I maintainer di solito rispondono entro 7 giorni
Check the date for the Pre-FOSDEM EESSI Hackathon in training-events/index.md
Aperta
- Qualcosa non va?
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 88/100

EESSI/docs#949 · 1 commento · Aperta il 8/11/2026

I maintainer di solito rispondono entro 1 giorno

Altre issue su Python

LLM Quantization Cannot Load `text_dataset` (Default and Custom Values Fail)

Qualcosa non va?

Valutazione

Direzione di ricerca

Descrizione

Describe the bug

Steps/Code to reproduce bug

System information (Nvidia Jetson Thor)

Preparare l'ambiente

Come iniziare

Altre issue di NVIDIA/TensorRT-Edge-LLM

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Issue simili

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Ricevi le nuove issue nella tua casella