Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

RemoteFortressReader: GetCreatureRaws sends CP437 (not UTF-8) strings, which breaks protobuf decoding

Ouverte
  • Quelque chose ne va pas ?
#6,019 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 3 jours

@dwmuller y travaille déjà.

Depuis le 20/10/2026.

  • #6021 par @dwmuller — ouverte

Évaluation

Difficulté
2/5
Temps estimé
1-3 heures
Accessibilité débutants
20/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
À l'abandon
Stack technique
cpp
Domaine
api, backend

Piste de recherche

Read plugins/remotefortressreader/remotefortressreader.cpp and check the creature-raw string fields, starting with the caste description assignment identified in the issue. Convert game-sourced strings to UTF-8 consistently, then validate GetCreatureRaws replies with strict UTF-8 decoding. A linked pull request is already open, so check its changes and status before taking this on.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Disclaimer: this issue was drafted by GitHub Copilot (an AI assistant) under the direction of user @dwmuller. The test results below come from real runs against a live game, but the source-code analysis is AI-generated, so please verify it.

Summary

RemoteFortressReader.GetCreatureRaws returns some strings in the game's CP437 encoding instead of UTF-8. Protobuf string fields must be UTF-8, so strict clients (Rust prost, and likely others) reject the whole reply. A single bad string makes the 23 MB reply undecodable.

The 50.08-r2 changelog says "RemoteFortressReader: ensured names are transmitted in UTF-8 instead of CP437", but DF2UTF is applied selectively and was not applied here.

Environment
  • Dwarf Fortress 53.16
  • DFHack 53.16-r3
  • Remote client: Rust dfhack-remote 0.12.0 (prost)
  • Fortress with many generated creatures (raw indices ~2026-2085 are affected)
Observed

Decoding fails with:

failed to decode Protobuf message: CasteRaw.description: CreatureRaw.caste:
CreatureRawList.creature_raws: invalid string value: data is not UTF-8 encoded.

I walked the raw reply bytes without UTF-8 validation and found 74 non-UTF-8 strings. They fall into three fields:

Field Count Example (raw bytes)
CreatureRaw.name (field 3) 19 Being of r\x86sh
CreatureRaw.caste[].caste_name (path 1.10.3) 31 Being of r\x86sh
CreatureRaw.caste[].description (path 1.10.15) 24 ...created by the dwarven goddess R\x86sh and is of a part with deformity...

0x86 is "å" in CP437, which fits these generated names ("Råsh").

Paging GetPartialCreatureRaws and bisecting shows that exactly 14 raws in the 2026-2085 range fail. Indices before 2026 decode fine.

Cause (from reading the source)

In plugins/remotefortressreader/remotefortressreader.cpp, the caste loop has:

send_caste->set_description(orig_caste->description);

There is no DF2UTF(...). The CreatureRaw/CasteRaw name fields (name, caste_name, baby/child names) show the same symptom in the data and should be checked as well. I only confirmed the unconverted set_description call in the source.

Not affected in this fortress
  • GetUnitList and GetUnitListInside (305 units) have no invalid UTF-8.
  • GetMaterialList, GetGrowthList, GetItemList and GetPlantRaws have no invalid UTF-8.

I looked at these specifically because they also carry caste strings. They're fine here, but it's worth auditing every string that reaches a protobuf message.

Suggested fix

Wrap all game-sourced strings in the creature-raw path with DF2UTF, and audit the other set_* string calls in the plugin for the same omission.

Reproduction

Any client works. Call GetCreatureRaws and validate the reply strictly as UTF-8. I wrote a small test harness that does this. It's in a project of mine, and everything is read-only:

https://github.com/dwmuller/DFCompanion/blob/experiment/dfhack-lua-http-spike/tests/live_game_tests.rs

Run it against a live game (DFHack remote on port 5000):

cargo test --test live_game_tests -- --ignored --nocapture --test-threads=1
  • creature_raws and creature_raws_paged use the typed client, and the paged one bisects to the failing indices.
  • raw_creature_raws_utf8 bypasses prost, reads the protobuf wire format directly, and lists every non-UTF-8 field by path with example bytes.
  • The other raw_* tests do the same for the unit and material/item lists.
Langage dominant
C++
Étoiles
2.1k
Forks
509
Merge moyen
2 j 19 h
PR mergées (30 j)
25

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de DFHack/dfhack

Toutes les issues de DFHack/dfhack

Issues similaires

Plus d'issues C++

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.