jingyaogong/minimind

如何让模型学会100内的加减法？

Open

#620 opened on Jan 7, 2026

View on GitHub
 (18 comments) (0 reactions) (0 assignees)Python (49,852 stars) (6,338 forks)batch import
help wantedquestion

Description

已经在104M的模型下sft了很多轮，数据格式如下： {"conversations": [{"role": "user", "content": "81 + 14="}, {"role": "assistant", "content": "95"}]} {"conversations": [{"role": "user", "content": "94 + 35="}, {"role": "assistant", "content": "129"}]} {"conversations": [{"role": "user", "content": "28 + 17="}, {"role": "assistant", "content": "45"}]} {"conversations": [{"role": "user", "content": "86 + 94="}, {"role": "assistant", "content": "180"}]} {"conversations": [{"role": "user", "content": "75 + 54="}, {"role": "assistant", "content": "129"}]} {"conversations": [{"role": "user", "content": "3 + 11="}, {"role": "assistant", "content": "14"}]} {"conversations": [{"role": "user", "content": "29 + 64="}, {"role": "assistant", "content": "93"}]} {"conversations": [{"role": "user", "content": "71 - 25="}, {"role": "assistant", "content": "46"}]} {"conversations": [{"role": "user", "content": "28 - 57="}, {"role": "assistant", "content": "-29"}]} {"conversations": [{"role": "user", "content": "0 + 97="}, {"role": "assistant", "content": "97"}]} {"conversations": [{"role": "user", "content": "89 - 54="}, {"role": "assistant", "content": "35"}]} {"conversations": [{"role": "user", "content": "35 + 19="}, {"role": "assistant", "content": "54"}]} {"conversations": [{"role": "user", "content": "97 + 43="}, {"role": "assistant", "content": "140"}]} {"conversations": [{"role": "user", "content": "11 + 48="}, {"role": "assistant", "content": "59"}]} {"conversations": [{"role": "user", "content": "45 - 44="}, {"role": "assistant", "content": "1"}]} {"conversations": [{"role": "user", "content": "5 - 93="}, {"role": "assistant", "content": "-88"}]} {"conversations": [{"role": "user", "content": "68 - 15="}, {"role": "assistant", "content": "53"}]} {"conversations": [{"role": "user", "content": "10 - 70="}, {"role": "assistant", "content": "-60"}]} {"conversations": [{"role": "user", "content": "80 - 79="}, {"role": "assistant", "content": "1"}]} {"conversations": [{"role": "user", "content": "73 + 24="}, {"role": "assistant", "content": "97"}]} 。。。。。 大概有10W条

效果依然不好

Contributor guide

Tech stack
python
Domain
machine learningai
Issue type
research
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
supervised fine tuninglanguage model trainingarithmetic data representation
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Investigate whether the model's tokenizer properly handles numeric tokens (e.g., splitting '81+14=' into meaningful subwords). Consider breaking down arithmetic into step by step reasoning or using a chain of thought format. Evaluate if the 64M parameter capacity is sufficient for this task, possibly increasing model size or using a different loss weighting for numeric answers.