Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

RemoteFortressReader: GetCreatureRaws sends CP437 (not UTF-8) strings, which breaks protobuf decoding

Aperta
  • Qualcosa non va?
#6,019 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@dwmuller ci sta già lavorando.

Dal 20/10/2026.

  • #6021 di @dwmuller — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
20/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
cpp
Ambito
api, backend

Direzione di ricerca

Read plugins/remotefortressreader/remotefortressreader.cpp and check the creature-raw string fields, starting with the caste description assignment identified in the issue. Convert game-sourced strings to UTF-8 consistently, then validate GetCreatureRaws replies with strict UTF-8 decoding. A linked pull request is already open, so check its changes and status before taking this on.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Disclaimer: this issue was drafted by GitHub Copilot (an AI assistant) under the direction of user @dwmuller. The test results below come from real runs against a live game, but the source-code analysis is AI-generated, so please verify it.

Summary

RemoteFortressReader.GetCreatureRaws returns some strings in the game's CP437 encoding instead of UTF-8. Protobuf string fields must be UTF-8, so strict clients (Rust prost, and likely others) reject the whole reply. A single bad string makes the 23 MB reply undecodable.

The 50.08-r2 changelog says "RemoteFortressReader: ensured names are transmitted in UTF-8 instead of CP437", but DF2UTF is applied selectively and was not applied here.

Environment
  • Dwarf Fortress 53.16
  • DFHack 53.16-r3
  • Remote client: Rust dfhack-remote 0.12.0 (prost)
  • Fortress with many generated creatures (raw indices ~2026-2085 are affected)
Observed

Decoding fails with:

failed to decode Protobuf message: CasteRaw.description: CreatureRaw.caste:
CreatureRawList.creature_raws: invalid string value: data is not UTF-8 encoded.

I walked the raw reply bytes without UTF-8 validation and found 74 non-UTF-8 strings. They fall into three fields:

Field Count Example (raw bytes)
CreatureRaw.name (field 3) 19 Being of r\x86sh
CreatureRaw.caste[].caste_name (path 1.10.3) 31 Being of r\x86sh
CreatureRaw.caste[].description (path 1.10.15) 24 ...created by the dwarven goddess R\x86sh and is of a part with deformity...

0x86 is "å" in CP437, which fits these generated names ("Råsh").

Paging GetPartialCreatureRaws and bisecting shows that exactly 14 raws in the 2026-2085 range fail. Indices before 2026 decode fine.

Cause (from reading the source)

In plugins/remotefortressreader/remotefortressreader.cpp, the caste loop has:

send_caste->set_description(orig_caste->description);

There is no DF2UTF(...). The CreatureRaw/CasteRaw name fields (name, caste_name, baby/child names) show the same symptom in the data and should be checked as well. I only confirmed the unconverted set_description call in the source.

Not affected in this fortress
  • GetUnitList and GetUnitListInside (305 units) have no invalid UTF-8.
  • GetMaterialList, GetGrowthList, GetItemList and GetPlantRaws have no invalid UTF-8.

I looked at these specifically because they also carry caste strings. They're fine here, but it's worth auditing every string that reaches a protobuf message.

Suggested fix

Wrap all game-sourced strings in the creature-raw path with DF2UTF, and audit the other set_* string calls in the plugin for the same omission.

Reproduction

Any client works. Call GetCreatureRaws and validate the reply strictly as UTF-8. I wrote a small test harness that does this. It's in a project of mine, and everything is read-only:

https://github.com/dwmuller/DFCompanion/blob/experiment/dfhack-lua-http-spike/tests/live_game_tests.rs

Run it against a live game (DFHack remote on port 5000):

cargo test --test live_game_tests -- --ignored --nocapture --test-threads=1
  • creature_raws and creature_raws_paged use the typed client, and the paged one bisects to the failing indices.
  • raw_creature_raws_utf8 bypasses prost, reads the protobuf wire format directly, and lists every non-UTF-8 field by path with example bytes.
  • The other raw_* tests do the same for the unit and material/item lists.
Lingua principale
C++
Stelle
2.1k
Fork
509
Merge medio
2g 13h
PR unite (30g)
28

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di DFHack/dfhack

Tutte le issue di DFHack/dfhack

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.