Model Hallucination Occurs again and again.

Reproduce el informe con setup_env.py y run_inference.py utilizando models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf y el prompt proporcionado. Comienza capturando la respuesta completa generada y los detalles del entorno; actualmente, el issue no define la salida esperada ni muestra la supuesta alucinación. Se considera terminado cuando el fallo sea reproducible y se hayan identificado claramente su causa y su corrección.

python -m venv venv
╰─❮ pip install -r requirements.txt
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
python-xyz', where xyz is the package you are trying to
install.

If you wish to install a non-Arch-packaged Python package,
create a virtual environment using 'python -m venv path/to/venv'.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.

If you wish to install a non-Arch packaged Python application,
it may be easiest to use 'pipx install xyz', which will manage a
virtual environment for you. Make sure you have python-pipx
installed via pacman.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
╰─❮ source venv/bin/activate.fish
╰─❮ pip install -r requirements.txt
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu, https://download.pytorch.org/whl/nightly, https://download.pytorch.org/whl/cpu, https://download.pytorch.org/whl/nightly, https://download.pytorch.org/whl/cpu, https://download.pytorch.org/whl/nightly
Ignoring torch: markers 'platform_machine == "s390x"' don't match your environment
Ignoring torch: markers 'platform_machine == "s390x"' don't match your environment
Collecting numpy~=1.26.4 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 1))
Using cached numpy-1.26.4-cp314-cp314-linux_x86_64.whl
Collecting sentencepiece<0.3.0,>=0.1.98 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 2))
Using cached sentencepiece-0.2.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (33 kB)
Collecting transformers==4.57.6 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached transformers-4.57.6-py3-none-any.whl.metadata (43 kB)
Collecting gguf>=0.1.0 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 6))
Using cached gguf-0.19.0-py3-none-any.whl.metadata (4.1 kB)
Collecting protobuf<5.0.0,>=4.21.0 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 7))
Using cached protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Collecting torch==2.11.0 (from -r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached torch-2.11.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (29 kB)
Collecting filelock (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached filelock-3.32.2-py3-none-any.whl.metadata (2.0 kB)
Collecting huggingface-hub<1.0,>=0.34.0 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached huggingface_hub-0.36.2-py3-none-any.whl.metadata (15 kB)
Collecting packaging>=20.0 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Downloading packaging-26.3-py3-none-any.whl.metadata (3.5 kB)
Collecting pyyaml>=5.1 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.4 kB)
Collecting regex!=2019.12.17 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached regex-2026.7.19-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
Collecting requests (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached requests-2.34.2-py3-none-any.whl.metadata (4.8 kB)
Collecting tokenizers<=0.23.0,>=0.22.0 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached tokenizers-0.22.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.3 kB)
Collecting safetensors>=0.4.3 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)
Collecting tqdm>=4.27 (from transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached tqdm-4.70.0-py3-none-any.whl.metadata (57 kB)
Collecting typing-extensions>=4.10.0 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
Collecting setuptools<82 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached setuptools-81.0.0-py3-none-any.whl.metadata (6.6 kB)
Collecting sympy>=1.13.3 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Collecting networkx>=2.5.1 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached networkx-3.6.1-py3-none-any.whl.metadata (6.8 kB)
Collecting jinja2 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec>=0.8.5 (from torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached fsspec-2026.7.0-py3-none-any.whl.metadata (10 kB)
Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub<1.0,>=0.34.0->transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Downloading hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (4.9 kB)
Collecting charset_normalizer<4,>=2 (from requests->transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (41 kB)
Collecting idna<4,>=2.5 (from requests->transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached idna-3.18-py3-none-any.whl.metadata (6.1 kB)
Collecting urllib3<3,>=1.26 (from requests->transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached urllib3-2.7.0-py3-none-any.whl.metadata (6.9 kB)
Collecting certifi>=2023.5.7 (from requests->transformers==4.57.6->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_legacy_llama.txt (line 4))
Using cached certifi-2026.7.22-py3-none-any.whl.metadata (2.5 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch==2.11.0->-r /home/luca/bitnet/BitNet/3rdparty/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt (line 5))
Using cached markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
Using cached transformers-4.57.6-py3-none-any.whl (12.0 MB)
Using cached torch-2.11.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl (190.3 MB)
Using cached sentencepiece-0.2.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB)
Using cached protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl (295 kB)
Using cached huggingface_hub-0.36.2-py3-none-any.whl (566 kB)
Downloading hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 4.3 MB/s 0:00:01
Using cached setuptools-81.0.0-py3-none-any.whl (1.1 MB)
Using cached tokenizers-0.22.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
Using cached gguf-0.19.0-py3-none-any.whl (118 kB)
Using cached fsspec-2026.7.0-py3-none-any.whl (206 kB)
Using cached networkx-3.6.1-py3-none-any.whl (2.1 MB)
Downloading packaging-26.3-py3-none-any.whl (129 kB)
Using cached pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (794 kB)
Using cached regex-2026.7.19-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (801 kB)
Using cached requests-2.34.2-py3-none-any.whl (73 kB)
Using cached charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (223 kB)
Using cached idna-3.18-py3-none-any.whl (65 kB)
Using cached urllib3-2.7.0-py3-none-any.whl (131 kB)
Using cached certifi-2026.7.22-py3-none-any.whl (136 kB)
Using cached safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (516 kB)
Using cached sympy-1.14.0-py3-none-any.whl (6.3 MB)
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Using cached tqdm-4.70.0-py3-none-any.whl (80 kB)
Using cached typing_extensions-4.16.0-py3-none-any.whl (45 kB)
Using cached filelock-3.32.2-py3-none-any.whl (98 kB)
Using cached jinja2-3.1.6-py3-none-any.whl (134 kB)
Using cached markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23 kB)
Installing collected packages: mpmath, urllib3, typing-extensions, tqdm, sympy, setuptools, sentencepiece, safetensors, regex, pyyaml, protobuf, packaging, numpy, networkx, MarkupSafe, idna, hf-xet, fsspec, filelock, charset_normalizer, certifi, requests, jinja2, torch, huggingface-hub, gguf, tokenizers, transformers
Successfully installed MarkupSafe-3.0.3 certifi-2026.7.22 charset_normalizer-3.4.9 filelock-3.32.2 fsspec-2026.7.0 gguf-0.19.0 hf-xet-1.6.0 huggingface-hub-0.36.2 idna-3.18 jinja2-3.1.6 mpmath-1.3.0 networkx-3.6.1 numpy-1.26.4 packaging-26.3 protobuf-4.25.9 pyyaml-6.0.3 regex-2026.7.19 requests-2.34.2 safetensors-0.8.0 sentencepiece-0.2.2 setuptools-81.0.0 sympy-1.14.0 tokenizers-0.22.2 torch-2.11.0+cpu tqdm-4.70.0 transformers-4.57.6 typing-extensions-4.16.0 urllib3-2.7.0

[notice] A new release of pip is available: 26.1.2 -> 26.2.1
[notice] To update, run: pip install --upgrade pip
╰─❮ python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s
INFO:root:Compiling the code using CMake.
INFO:root:Loading model from directory models/BitNet-b1.58-2B-4T.
INFO:root:GGUF model already exists at models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf
╰─❮ python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "how are you, are you good?"
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions

Loading model...

▄▄ ▄▄
██ ██
██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
██ ██
▀▀ ▀▀

build : b9918-390c30775
model : models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf
ftype : Q1_0
modalities : text

available commands:
/exit or Ctrl+C stop or exit
/regen regenerate the last response
/clear clear the chat history
/read add a text file
/glob add text files using globbing pattern

how are you, are you good?

Yes, I am good. I am a good person. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes, I am good. Yes,

[ Prompt: 44.9 t/s | Generation: 11.7 t/s ]