facebookresearch/blt

View on GitHub

Code for BLT research paper

Stars: 2.1k
Forks: 196
Open beginner issues: 0
Indexed issues: 46

Dominant language: Python
License: No license data
Last GitHub push: Nov 3, 2025
Latest indexed: Sep 15, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

46 open issues indexed

Appendix O: Entropy Model Dimensions Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

facebookresearch/blt#143 · Opened Jul 13, 2026
Great!!! And a question: What is the best tokenization-free algorihtm? SpaceByte, ByteLatentTransformer, DynamicChunking@SSM? Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

facebookresearch/blt#142 · Opened May 26, 2026
Why is the average patch size inconsistent with the value reported in the paper? Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

facebookresearch/blt#141 · Opened Apr 20, 2026
cuda 13 support Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#138 · 1 comment · 1 reaction · Opened Jan 4, 2026
demo.py RuntimeError: CUDA error: an illegal memory access was encountered Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#136 · 2 comments · 1 reaction · Opened Oct 16, 2025
How to reset entropy context -- avoiding entropy "drifting"? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#135 · Opened Sep 13, 2025
When bpe_delim = True, do we skip BPE_ID when calculating NLL? Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

facebookresearch/blt#134 · Opened Sep 8, 2025
Unclear how to preprocess entropy data Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

facebookresearch/blt#133 · 2 reactions · Opened Sep 1, 2025
LMTransformer.__init__() missing 1 required positional argument: 'args' Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

facebookresearch/blt#131 · 7 comments · Opened Jul 3, 2025
ModuleNotFoundError: No module named 'setuptools' Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

facebookresearch/blt#130 · Opened Jul 3, 2025
Why do you require that first patch should have one non-boe token? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#129 · Opened Jun 30, 2025
Validation Loss Stagnate on BLT Arch Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

facebookresearch/blt#128 · Opened Jun 29, 2025
Cannot Reproduce BLT Tokenization with bpe_delim=True Due to Missing BPE Model File Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#125 · 2 comments · Opened Jun 5, 2025
Triton compilation error occurred during inference Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#124 · Opened Jun 5, 2025
Patching code unsuitable for batch inference Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

facebookresearch/blt#123 · Opened Jun 3, 2025
Architectural clarifications, decoder's halting condition Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

facebookresearch/blt#121 · 1 reaction · Opened Jun 1, 2025
Where can I find the .pt files needed for testing? Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

facebookresearch/blt#119 · 1 comment · Opened May 21, 2025
Multilingual Support Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#118 · 2 comments · Opened May 21, 2025
Understanding scaling trends & inference code question Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

facebookresearch/blt#114 · 2 comments · Opened May 19, 2025
Why use cross_attn_k Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

facebookresearch/blt#109 · 1 reaction · Opened May 15, 2025
Integration with HF: tokenizer Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

facebookresearch/blt#108 · 4 comments · Opened May 12, 2025
How to choose the appropriate patch threshold for a new task? Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

facebookresearch/blt#106 · 1 comment · Opened May 8, 2025
A question about training process of Entropy Model. Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#100 · Opened Apr 22, 2025
Clarification on patch query initialization in local encoder Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#88 · 3 comments · Opened Apr 1, 2025
torch/dynamo/torch.compile causes triton.compiler.errors.CompilationError Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#87 · 6 comments · Opened Mar 25, 2025
FLOPs calculation Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#78 · 2 comments · Opened Mar 10, 2025
GPU Budget Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

facebookresearch/blt#70 · 1 comment · 3 reactions · Opened Feb 24, 2025
KV Cache in BLT Generation. Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#67 · 3 comments · 1 reaction · Opened Feb 21, 2025
Hash embeddings abusing Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#64 · 4 comments · Opened Feb 20, 2025
Mistake in FLOPs calculations in the paper Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#61 · 3 comments · 1 reaction · Opened Feb 15, 2025
What is BOE_ID? Open

Difficulty 1/5 Under an hour Newbie friendliness 30/100

facebookresearch/blt#59 · 2 comments · Opened Feb 14, 2025
How to use the BLT model and attach it to some existing transformer? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

facebookresearch/blt#51 · 2 comments · Opened Feb 12, 2025
BLT Inference Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#37 · 3 comments · 1 reaction · Opened Feb 3, 2025
Current implementation of patcher does not support sequence packing Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

facebookresearch/blt#36 · 4 comments · Opened Feb 1, 2025
What does Local Decoder return? Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

facebookresearch/blt#30 · 16 comments · Opened Jan 20, 2025
Question about Encoder Hash n-gram Embeddings Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

facebookresearch/blt#29 · 2 comments · Opened Jan 19, 2025
Question about an implementation in packing_iterator.py. Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

facebookresearch/blt#27 · 1 comment · Opened Jan 18, 2025
What are the patch lengths and the initial values of the generated tokens during generation? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#24 · 33 comments · Opened Jan 16, 2025
Missing script to train the entropy model Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

facebookresearch/blt#20 · 1 comment · Opened Jan 13, 2025
Share example data for test_blt.py Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

facebookresearch/blt#15 · 2 comments · 1 reaction · Opened Jan 7, 2025
Question about (5)equation Open

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

facebookresearch/blt#14 · Opened Jan 3, 2025
Question about math/arithmetic Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

facebookresearch/blt#12 · Opened Dec 30, 2024
Files for Running Test Cases Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

facebookresearch/blt#10 · 1 comment · Opened Dec 26, 2024
.DS_Store file is must we need? Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

facebookresearch/blt#8 · 1 comment · 1 reaction · Opened Dec 20, 2024
Cross Attention does not work on CPU and older GPUs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

facebookresearch/blt#6 · 6 comments · Opened Dec 18, 2024
Release Byte Latent Transformer (BLT) on Hugging Face Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

facebookresearch/blt#5 · 11 comments · 20 reactions · Opened Dec 18, 2024

facebookresearch/blt

Get new issues in your inbox