I installed it on Ubuntu Linux. Here some common help with that...

Xem lại hướng dẫn Ubuntu 24.04 được đề xuất cho docs/ubuntu-24.04-guide.md, sau đó xác minh setup_env.py, src/ggml-bitnet-mad.cpp và các lệnh build trên Ubuntu. Công việc được xem là hoàn tất khi một hướng dẫn tập trung ghi lại chính xác các bước về Clang, CMake, const-correctness, tải xuống model, build và inference.

Here's a clear, structured summary you can copy-paste (or slightly adapt) into a GitHub issue, discussion, wiki page, or README section like “Running on Ubuntu 24.04 – Common Pitfalls & Fixes”.

Feel free to change the tone, add your username, screenshots or exact dates if you want.

markdown

Running Microsoft BitNet b1.58-2B-4T on Ubuntu 24.04 (Noble Numbat) – Experience & Fixes

March 2026 – tested on Ubuntu 24.04 LTS with fresh install + many PPAs already present.

Goal: run the official repo → https://github.com/microsoft/BitNet
Model: ggml-model-i2_s.gguf from Hugging Face microsoft/BitNet-b1.58-2B-4T-gguf

Summary of problems we hit & how we solved them

1. Installing recent Clang (needed ≥18, better ≥19–20)

sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
sudo apt update
sudo apt install clang-20 lld-20 cmake

→ installs clang-20 and clang++-20 (not plain clang / clang++)

2. CMake cannot find compiler → "clang is not a full path and was not found in the PATH"

Cause
setup_env.py hard-codes -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
→ looks for unversioned clang, but Ubuntu llvm.sh only creates versioned binaries.

Quick & clean workaround (worked best for us):

mkdir -p ~/bin
ln -s /usr/bin/clang-20   ~/bin/clang
ln -s /usr/bin/clang++-20  ~/bin/clang++

# Add to current session (or put in ~/.bashrc)
export PATH="$HOME/bin:$PATH"

After this which clang → /home/yourname/bin/clang and clang --version shows 20.x

3. Build fails in ggml-bitnet-mad.cpp with const-correctness error (Clang 20)

Error (line ~811):

cannot initialize a variable of type 'int8_t *' with an rvalue of type 'const int8_t *'
   int8_t * y_col = y + col * by;

Cause
Clang 20 is stricter about discarding const qualifiers than Clang 18 or GCC.

Fix (minimal patch – safe because pointer is only read from)

Edit file:

nano src/ggml-bitnet-mad.cpp

Change line ~811 from:

int8_t * y_col = y + col * by;

to:

const int8_t * y_col = y + col * by;

Save → re-run

python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s
# or directly:
cmake --build build --config Release -j$(nproc)

Alternative: use Clang 18 (less strict)

sudo apt install clang-18 lld-18
rm ~/bin/clang ~/bin/clang++
ln -s /usr/bin/clang-18 ~/bin/clang
ln -s /usr/bin/clang++-18 ~/bin/clang++
export PATH="$HOME/bin:$PATH"
rm -rf build
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

4. Final working flow (after fixes)

# inside venv, in BitNet repo root
export PATH="$HOME/bin:$PATH"   # if not in .bashrc yet

# Download model if not already there
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf \
  ggml-model-i2_s.gguf --local-dir models/BitNet-b1.58-2B-4T

# Build (with the symlink trick + patch applied)
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

# Run chat
python run_inference.py \
  -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf \
  -p "You are a helpful assistant" \
  -cnv

Tips / Notes

• Model uses ~450–600 MB RAM → excellent for laptops / low-power machines
• Pure CPU right now (GPU support for i2_s quantization is still experimental in March 2026)
• If you want to try GPU later → build recent llama.cpp with -DLLAMA_CUDA=ON and test the same .gguf file with --n-gpu-layers 999
• Many warnings during compile (missing prototypes, unused params, double→float) are harmless

Hope this saves someone a few hours of frustration.

Tested on: Ubuntu 24.04, Ryzen CPU, Clang 20 from apt.llvm.org
Good luck & enjoy the tiny-but-surprisingly-useful model! 🚀

Feel free to add your hardware specs, approximate tokens/s, or any extra observations.

You can put this in:

- `docs/ubuntu-24.04-guide.md`
- a pinned Discussion thread
- or even as a new issue titled “Ubuntu 24.04 installation guide – common pitfalls & solutions”

Let me know if you want it shorter, more formal, or with extra sections (e.g. GPU attempt notes). Congrats again on getting it running!