[Benchmark] GTX 1070 Ti 8 GB (sm_61), Gyro-S: works, and the VRAM reserve is what decides it

Start with docs/OLDER_GPUS.md, BENCHMARK_PROTOCOL.md, and MODELS.md; compare their existing guidance with the GTX 1070 Ti measurements and VRAM reserve warning in this report. Document the reserve and stalled requests, the larger-context result, and the role of the conversation cache where appropriate. Done when the requested guidance is consistent across the relevant docs and clearly distinguishes measured results from suggestions.

Summary

docs/OLDER_GPUS.md says under the P40 section: "Not tested: any other sm_61 card (Tesla P4,
GTX 10 series)." This is that data point, and it grew a finding I did not expect.

A GTX 1070 Ti (8 GB, compute capability 6.1) runs Gyro-S on the released CUDA 12 engine. It
passes the standard protocol (21 of 21 required requests, 12 of 12 sanity checks), it
completes real multi-turn Hermes agent tasks with tool calls at a perfect score, and it loads in
13 to 18 seconds from a local NVMe.

The single most useful thing I can report is not a throughput number. It is that a request
which never returns is almost always the VRAM reserve, not the model, and your own log says so.
Details below. It cost me several hours of chasing page cache, swap and swappiness, all of which
looked plausible and were wrong.

Hardware and software

GPU	NVIDIA GTX 1070 Ti, 8 GB, compute capability 6.1
Driver	580.173.02, CUDA 12.9 via the released ...-cuda-cuda12 engine
CPU	AMD Ryzen 9 3900X, 12 cores / 24 threads, AVX2, no AVX-512
RAM	32 GB DDR4, 16 GB zram swap (zstd)
Storage	Samsung 990 EVO+ NVMe, model files local
OS	Pop!_OS 24.04, kernel 6.17

Strata Terminal 0.2.7, engine 0.1.52 linux-x86_64-cuda-cuda12, released binary.
Model gyro Gyro-S (Qwen3.8-Flash-Next trellis pack), 58.5 GB plus the MTP layer, vision off.
32 GB of RAM is below the 64 GB your matrices recommend, so setup chose low-memory mode.

1. The VRAM reserve, which is the real finding

On this card the engine leaves very little VRAM free, and below a 512 MiB reserve the NVIDIA
driver pages GPU memory to system RAM. The request then does a burst of work and stops, with no
error, no CPU load, no GPU load, and no result. Your log states it plainly:

WARNING: only 218 MiB of VRAM free at the start of this request (below 512 MiB):
when the card is this full the driver can page GPU memory to system RAM, decode falls to a few
tok/s and a window may never finish (#2065).
Add --vram-reserve-mib 1344 to the config's args, or lower --max-context / --batch

Observed at 64K with no reserve: three separate requests produced no result after 150, 300 and
540 seconds. Host side, throughout: 22 GB of memory available, swap quiet, GPU spiking to 60 or
80 percent and then idle. Not a memory problem, not a model problem, and not a hang. The
suggestion in the warning is the fix, and it worked first time:

	free VRAM at load	10K prompt result
no reserve	178 to 505 MiB	no result in 150 s, twice
--vram-reserve-mib 1024	1,482 MiB	completed in 97 s
--vram-reserve-mib 1344	1,164 MiB	stable

Two suggestions for the docs: the reserve needs to scale with --max-context rather than being a
flat number, because at 48K a 1024 MiB reserve is generous and at 64K it is not enough (218 MiB
free was still measured with 1024 set at 64K). And a request that returns nothing at all is a
distinct outcome from a slow one, so it deserves a line in BENCHMARK_PROTOCOL.md; otherwise a
single-request measurement on a small card reports noise.

2. Prompt throughput, all at one fixed reserve of 1024 MiB

Unique prompts, max_tokens: 1, non-streaming, usage.prompt_tokens divided by wall time, three
warm-up requests discarded. tok/s.

Prompt	8K / int8	32K / q4_0 no reserve	32K / q4_0	48K / q4_0	64K / q4_0
~4.6K warm	203	203	133	168	89
~9K	211	211	138	176	89
~18K	211	211	144	173	91
~27K	n/a	211	145	173	93
~37K	n/a	n/a	n/a	105	93

Decode is 27 to 34 tok/s across all of them and is not affected by the window. Prompt speed is.

The reserve is not free, and I originally reported the wrong figure. My first report said
32K is "free". It was measured without a reserve, at 178 to 505 MiB free, which is inside the
stall band. With the reserve it is 133 to 145 tok/s. Please treat the earlier number as unsafe
rather than as a better result.

3. A larger window can be faster than a smaller one

This surprised me enough to run it twice. At the same 1024 MiB reserve, 48K beats 32K at every
prompt length that both can hold:

Prompt	32K + reserve	48K + reserve
~9K	138	176
~18K	144	173
~27K	145	173

The reason looks like arithmetic rather than anything clever: the reserve is a fixed cost, and
it is taken out of expert slots. At 48K the KV cache is large enough that the reservation
displaces proportionally fewer slots, so the expert cache survives it better. If that holds
generally, a flat --vram-reserve-mib penalises small windows disproportionately, and the
guidance "take the largest context the card will accept" is the right advice for a small card.

4. The expert set does not fit the page cache, and that is the real prompt bottleneck

prefill plan: chunk 1280, ring 96 kept: file set 24.2 GiB does not fit the page cache 21.3 GiB
-> about 518.7 GB read from the drive per 32768-token prompt (26 chunks)
no option does better (the set cannot fit: fewest passes win)

That is your planner's own summary and it is the clearest explanation of small-card behaviour I
have seen. It also tells me --prefill tuning is pointless here, because the fewest-passes
option was already chosen.

The consequence for a 32 GB box: page cache tops out near 21 GiB and the set needs 24.2 GiB, so
it misses by about three gigabytes and every chunk re-reads the lot. Doubling RAM to 64 GB would
make the set fit, which should turn a 26-pass sweep into roughly one pass, about 519 GB of reads
per prompt down to about 24 GB. That, not a flag, is the upgrade this configuration needs, and
it is worth saying in the docs for anyone choosing between more VRAM and more RAM on a small card.

5. It runs real agent work, not just a benchmark harness

Your protocol's sanity checks are single-shot. The question I actually had was whether it can
drive a multi-turn agent with tools. Tested with a twelve-task agent harness where the score is
the artifact left on disk, not the prose, at 64K with a 1344 MiB reserve:

Task	What it grades	Score	Turns	Tool calls	Wall
01-json-extract	contract, format, arithmetic over a CSV	1.00	12	10	340 s
04-rename-safe	performed the file operations, did not assert them	1.00	8	6	369 s
05-memory-nodestructive	restraint; input hashes unchanged	1.00	8	6	498 s
09-long-context-retrieval	facts pulled out of a large log	1.00	8	6	310 s

Four of four, no harness faults, every run exit 0. The conversation cache carried each task: 12
turns cost 133,065 input tokens, about 11K per turn, because the first turn's prompt is reused
rather than re-read. Without that cache this would not have been viable at all, so if it is
optional on other models it is load bearing here.

I have not yet run the harder three (03-test-fix, 10-multifile-refactor,
06-multiturn-continuity). Four passes is a first pass, not a suite, and I would not claim more.

6. One note for anyone using Strata as an agent backend

Not a Strata issue, but it decides the question: Hermes Agent refuses any model reporting a
context window below 64,000 tokens (MINIMUM_CONTEXT_LENGTH in agent/model_metadata.py, raised
in agent/agent_init.py), and there is no override for an OpenAI-compatible endpoint. So 8K,
32K and 48K cannot host a Hermes agent at any speed, however well they measure, and only 64K is
usable. The child exits in under ten seconds with zero tokens and a misleading
"Execution outcome is unknown" message; the real ValueError is in
~/.hermes/logs/managed-worker.log, not in the harness log. Worth knowing before someone
concludes the engine is broken.

Requests

• Document the reserve interaction, and consider scaling it with context by default.
• Document that a never-finishing request is the VRAM reserve and where to look for it, in
  OLDER_GPUS.md next to the sm_61 table.
• Say in MODELS.md that a small card should take the largest context it will accept, since a
  bigger window measured faster here, not slower.
• Keep the conversation cache loud in the docs. It is what makes agent loops possible on a slow
  card.

The README.md and results.json from the standard run are saved locally and I can open a
results pull request with them if that is more useful than this write-up.

Background load

COSMIC desktop on the same card. No other GPU work. Power limit stock. Host swap is 16 GB zram
with vm.swappiness at 180 while measuring, which I mention only because it turned out not to
matter and someone else may lose time to it.