Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

RemoteFortressReader: GetCreatureRaws sends CP437 (not UTF-8) strings, which breaks protobuf decoding

Đang mở
  • Có gì không ổn?
#6,019 3 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 3 ngày

@dwmuller đang làm issue này rồi.

Từ ngày 20/10/2026.

  • #6021 của @dwmuller — đang mở

Đánh giá

Độ khó
2/5
Thời gian dự kiến
1-3 giờ
Mức phù hợp với người mới
20/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
cpp
Lĩnh vực
api, backend

Hướng nghiên cứu

Read plugins/remotefortressreader/remotefortressreader.cpp and check the creature-raw string fields, starting with the caste description assignment identified in the issue. Convert game-sourced strings to UTF-8 consistently, then validate GetCreatureRaws replies with strict UTF-8 decoding. A linked pull request is already open, so check its changes and status before taking this on.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Disclaimer: this issue was drafted by GitHub Copilot (an AI assistant) under the direction of user @dwmuller. The test results below come from real runs against a live game, but the source-code analysis is AI-generated, so please verify it.

Summary

RemoteFortressReader.GetCreatureRaws returns some strings in the game's CP437 encoding instead of UTF-8. Protobuf string fields must be UTF-8, so strict clients (Rust prost, and likely others) reject the whole reply. A single bad string makes the 23 MB reply undecodable.

The 50.08-r2 changelog says "RemoteFortressReader: ensured names are transmitted in UTF-8 instead of CP437", but DF2UTF is applied selectively and was not applied here.

Environment
  • Dwarf Fortress 53.16
  • DFHack 53.16-r3
  • Remote client: Rust dfhack-remote 0.12.0 (prost)
  • Fortress with many generated creatures (raw indices ~2026-2085 are affected)
Observed

Decoding fails with:

failed to decode Protobuf message: CasteRaw.description: CreatureRaw.caste:
CreatureRawList.creature_raws: invalid string value: data is not UTF-8 encoded.

I walked the raw reply bytes without UTF-8 validation and found 74 non-UTF-8 strings. They fall into three fields:

Field Count Example (raw bytes)
CreatureRaw.name (field 3) 19 Being of r\x86sh
CreatureRaw.caste[].caste_name (path 1.10.3) 31 Being of r\x86sh
CreatureRaw.caste[].description (path 1.10.15) 24 ...created by the dwarven goddess R\x86sh and is of a part with deformity...

0x86 is "å" in CP437, which fits these generated names ("Råsh").

Paging GetPartialCreatureRaws and bisecting shows that exactly 14 raws in the 2026-2085 range fail. Indices before 2026 decode fine.

Cause (from reading the source)

In plugins/remotefortressreader/remotefortressreader.cpp, the caste loop has:

send_caste->set_description(orig_caste->description);

There is no DF2UTF(...). The CreatureRaw/CasteRaw name fields (name, caste_name, baby/child names) show the same symptom in the data and should be checked as well. I only confirmed the unconverted set_description call in the source.

Not affected in this fortress
  • GetUnitList and GetUnitListInside (305 units) have no invalid UTF-8.
  • GetMaterialList, GetGrowthList, GetItemList and GetPlantRaws have no invalid UTF-8.

I looked at these specifically because they also carry caste strings. They're fine here, but it's worth auditing every string that reaches a protobuf message.

Suggested fix

Wrap all game-sourced strings in the creature-raw path with DF2UTF, and audit the other set_* string calls in the plugin for the same omission.

Reproduction

Any client works. Call GetCreatureRaws and validate the reply strictly as UTF-8. I wrote a small test harness that does this. It's in a project of mine, and everything is read-only:

https://github.com/dwmuller/DFCompanion/blob/experiment/dfhack-lua-http-spike/tests/live_game_tests.rs

Run it against a live game (DFHack remote on port 5000):

cargo test --test live_game_tests -- --ignored --nocapture --test-threads=1
  • creature_raws and creature_raws_paged use the typed client, and the paged one bisects to the failing indices.
  • raw_creature_raws_utf8 bypasses prost, reads the protobuf wire format directly, and lists every non-UTF-8 field by path with example bytes.
  • The other raw_* tests do the same for the unit and material/item lists.
Ngôn ngữ chính
C++
Star
2.1k
Fork
509
Merge trung bình
2 ngày 19 giờ
Pull request đã merge (30 ngày)
25

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của DFHack/dfhack

Tất cả issue của DFHack/dfhack

Issue tương tự

Thêm issue về C++

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.