[Other] the assistant is never told which model it runs on, and answers "Claude" when asked

Open Beginner friendly
#5,971 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, nodejs

Research direction

The bad identity line is generated from the AGENT_PROMPT template in host/server/agent/preset.mjs (line 42), where {{model}} is filled by the harness from the API model id. The label is built in host/server/agent/profile.mjs near line 239 and is already available there as inst.label. Start by reading the profile model list and the preset prompt, then pass the friendly label in as its own prompt variable. Done means the persona prompt names the human-readable model while keeping the route id, and the session's system message shows that line.

Written by the indexing model from the issue text.

Description

Area

[Other] (the agent chat layer shipped inside the desktop app)

Desktop app, Terminal or source?

Desktop app (Windows)

What happened

In a desktop-app chat with the local model, I asked the assistant "what model are you?" (in Chinese: 你是什么模型?). It answered, verbatim from the transcript of that chat:

我是 Claude,由 Anthropic 开发的大语言模型。…("I am Claude, a large language model developed by Anthropic")
关于具体版本号,我这边没有可靠的显示信息…("about the exact version number I have no reliable information here")

Both parts are wrong for this setup: the chat was served by the local model the app shows as Qwen3.8-Flash-Next IQ3_S.

I checked what the assistant was actually told about its own identity, and the app tells it a quantization slug only. From this session's own transcript (C:\Users\123\.strata\data\agent\home\sessions\--C-Users-123-Documents-Strata-Workspace--\session-6d6296c6-b58c-4f2b-bfd1-66f074045ae3\session.v4.jsonl.zstd, decoded with node zlib.zstdDecompressSync, 48 frames), the system/message event at seq 9 contains the complete system prompt:

You are a helpful assistant that can also work on this computer through your tools. You are running as iq3_s.

That first line is the only thing in the prompt that speaks about the model's identity; the rest of the prompt is about the tools and the operating system. The model/selection and request/header events of the same session record the route: "provider": "strata-local", "model": "iq3_s", "reasoningEffort": "medium".

Where that text comes from, in the shipped desktop bundle (Strata 0.2.13, harness deepseek-harness 0.2.1-alpha.3), host/server/agent/preset.mjs:

// line 42
'You are a helpful assistant that can also work on this computer through your tools. You are running as {{model}}.',

// line 76
{ id: 'persona', name: '@deepseek-ai/dsh-persona', config: { prefix: agentPromptFor(platform), complete: true, includeRuntimeContext: false } },

{{model}} is resolved by the Harness, in node_modules/@deepseek-ai/dsh-agent-loop/lib/index.js:1591-1592:

ctx.systemPrompt.variable("provider", (context) => context.agent?.options.provider);
ctx.systemPrompt.variable("model", (context) => context.agent?.options.model);

So {{model}} is the API model id, and for a local instance that id is the catalog size slug. host/server/index.mjs:215-216 builds the local route from the loaded instance:

localInstances: () => native.readyInstances().map(i => ({
  id: i.id, label: i.label, baseUrl: `http://127.0.0.1:${i.port}/v1`, modelId: i.apiModelId,

and host/server/agent/profile.mjs:239 puts that id into the provider model list, keeping the friendly name beside it:

models: [{ id: inst.modelId, name: inst.label || inst.modelId, contextWindow: context, ... }],

inst.label is the name a person reads. host/server/model-names.mjs exists exactly for that purpose ("The name a person reads for a loaded model: the family and the version from the catalog … not the file id"), and host/server/native-control.mjs:477 sets it:

label: friendlyModelName(snapshot.catalog, args.model) ?? entry.model_name ?? args.model

On this machine that label is Qwen3.8-Flash-Next IQ3_S — it is what the load summary says in C:\Users\123\.strata\data\native-jobs.json ("Qwen3.8-Flash-Next IQ3_S is ready."), and it is what the model picker shows. So the app knows the human-readable model name at the moment it builds the agent profile, passes it into the route table (profile.mjs:247, name: inst.label || inst.modelId) and into the helper-agent tool text (host/server/agent/plugins/strata-models.mjs:27), and never puts it in front of the assistant that answers the person's questions.

The result is that the only identity string in the prompt is iq3_s. That is a quantization name, not a model name. In this app the size id is not unique to a family: native-jobs.json on this machine records an install of IQ3_S under family qwen and of IQ2_XS under family swift, and host/server/model-names.mjs exists precisely because the size id alone is not the name a person reads. So the slug cannot tell an assistant what model it is, and a small local model has nothing to answer "what model are you?" with except its training prior — which produced "Claude".

Expected: the assistant can name the model the person sees in the app. Actual: it invents one, and there is nothing in its prompt to stop it.

Smallest steps or command that reproduce it
  1. Desktop app 0.2.13 on Windows with a local model loaded (any size; here IQ3_S).
  2. Open a new chat, ask: "What model are you?" (in any language).
  3. The assistant names a different model family, or says it cannot know. The chat's own system prompt contains only You are running as <size slug>; to read that line back, decode the session transcript ($env:USERPROFILE\.strata\data\agent\home\sessions\<workspace>\session.v4.jsonl.zstd is a chain of zstd frames, one JSON line per event) with this read-only script:
// identity-line-check.mjs — prints the identity line and the route of one chat session
import { readFileSync } from 'node:fs';
import { zstdDecompressSync } from 'node:zlib';
const buf = readFileSync(process.argv[2]);
const magic = [0x28, 0xb5, 0x2f, 0xfd];
const starts = [];
for (let i = 0; i + 4 <= buf.length; i++) {
  if (buf[i] === magic[0] && buf[i + 1] === magic[1] && buf[i + 2] === magic[2] && (buf[i + 3] === 0xfd || buf[i + 3] === 0xfc)) starts.push(i);
}
const events = [];
for (let k = 0; k < starts.length; k++) {
  try {
    const text = zstdDecompressSync(buf.subarray(starts[k], k + 1 < starts.length ? starts[k + 1] : buf.length)).toString('utf8');
    for (const line of text.split('\n')) { if (line.trim()) { try { events.push(JSON.parse(line)); } catch {} } }
  } catch {}
}
const seen = new Set();
for (const e of events) {
  if (e.type !== 'system/message') continue;
  for (const part of e.data?.message?.content ?? []) {
    const m = /^You are [^\n]*running as (\S+?)\.?$/m.exec(part.text ?? '');
    if (m && !seen.has(m[0])) { seen.add(m[0]); console.log(`seq=${e.seq} ${m[0]}`); }
  }
}
for (const e of events) {
  if (e.type === 'model/selection') { console.log(`seq=${e.seq} model/selection ${JSON.stringify(e.data)}`); break; }
}

Ran here on three different chats of this workspace (node identity-line-check.mjs <session.v4.jsonl.zstd>), output verbatim:

--- session-6d6296c6-… ---   seq=9  You are a helpful assistant that can also work on this computer through your tools. You are running as iq3_s.
                            seq=4  model/selection {"provider":"strata-local","model":"iq3_s","reasoningEffort":"medium"}
--- session-08d60cd2-… ---   seq=15 You are a helpful assistant … You are running as iq3_s.
                            seq=4  model/selection {"provider":"strata-local","model":"iq3_s","reasoningEffort":"medium"}
--- session-8e9d53da-… ---   seq=15 You are a helpful assistant … You are running as iq3_s.
                            seq=4  model/selection {"provider":"strata-local","model":"iq3_s","reasoningEffort":"medium"}

So the prompt carries no model name in every chat, not only in the one that answered "Claude".

Suggested fix, smallest first: pass the friendly label through to the persona as its own prompt variable (the profile already has it) and use it in AGENT_PROMPT line 1, keeping the route id beside it so the person can still match what the model says to what the picker shows — for example You are running as Qwen3.8-Flash-Next IQ3_S (this chat's model id is iq3_s). Nothing else in the preset needs to change. Note complete: true at preset.mjs:76 also drops the Harness's own harness:identity section (node_modules/@deepseek-ai/dsh-system-prompt/src/index.ts:449-455), so this line really is the only identity the assistant has.

Reproduction status

Reproduced locally

Strata version

Strata 0.2.13 · engine 0.1.59-windows-x86_64-cuda-cuda13 · harness deepseek-harness 0.2.1-alpha.3

Graphics card, memory and system

NVIDIA GeForce RTX 3080 Ti Laptop GPU, 16 GB VRAM, driver 621.14 · 64 GB RAM · Windows 11 Pro 10.0.26300

Model, size and settings

Qwen3.8-Flash-Next IQ3_S · 524288 context chosen at setup, the session reported contextWindow: 393216 · pictures on · reasoning effort medium · one request at a time

Logs

Not attached: the failure is in what the assistant was told, not in the engine. The verbatim prompt, route and label are quoted above from the session transcript, native-jobs.json and the installed bundle's own files, and each can be re-read on any 0.2.13 install that has a local model loaded.

Dominant language
C++
Stars
11.6k
Forks
1k
Avg merge
18d 13h
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Niko1221/Strata

All issues in Niko1221/Strata

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.