ZenKit: mod world with 10/266/522 materials fails to load (std::bad_alloc) - Mroczne Tajemnice ANCIENTTEMPLE.ZEN

Aperta
#504 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp, unity
Ambito
game-dev

Direzione di ricerca

Start with the proposed ReadArchiveBinary::read_header fix in ZenKit and its binary header unit test; the affected file is src/archive/ArchiveBinary.cc. Validate the fix against a world with 266 materials and the listed LF, CRLF, and NUL header cases. Done means the material count is read correctly and the world loads; testing with the real ANCIENTTEMPLE.ZEN is still pending, and the ZenKit issue and pull request have not been submitted.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Problem

Mroczne Tajemnice (Gothic 1 mod) world ANCIENTTEMPLE.ZEN doesn't load: ZenKit throws std::bad_alloc while reading the world mesh. The original engine loads it fine.

Cause (ZenKit)

A world mesh's materials are a small binary archive inside the MeshAndBsp chunk (0xB020). Its header ends with objects N…\nEND\n\n, then the binary data starts with the material count.

ReadArchiveBinary::read_header (src/archive/ArchiveBinary.cc:25) reads the second END with read_line_then_ignore("\n"), which skips every \n after it - also the first data byte when it is 0x0A. That's the low byte of the material count for 10, 266, 522, … materials. ANCIENTTEMPLE.ZEN has exactly 266 materials, so everything after is read one byte off → std::bad_alloc.

Workaround we had (removed again, branch split/23-world)

ZenWorldArchiveFixDomain.TryFixMaterialCountHeader: when a world failed to load, ResourceCacheService.LoadWorld copied the world file, replaced the materials archive's END\n\n with END\0 (ends the line without skipping) and retried once.

Workaround code
    /// <summary>
    /// Workarounds for two ZenKit bugs with mod worlds, always on (ResourceCacheService.LoadWorld).
    /// FOLLOW-UP: raise both upstream (issue + PR at GothicKit/ZenKit), remove this class once a ZenKit release has
    /// the fixes.
    /// 1. Material count (TryFixMaterialCountHeader): a world mesh's materials are a small BINARY archive inside the
    ///    MeshAndBsp chunk. ZenKit reads its header's second "END" with read_line_then_ignore("\n"), which skips
    ///    every '\n' after it - also the first byte of the material count when it's 0x0A (10, 266, 522, ...
    ///    materials). The count and everything after is read one byte off and the loader dies with
    ///    std::bad_alloc (Mroczne Tajemnice's ANCIENTTEMPLE.ZEN has exactly 266). "END\0" ends the line without
    ///    skipping anything.
    /// 2. Chunk length (TryFixMeshChunkLengths): see there.
    /// </summary>
// ...
        public static byte[] TryFixMaterialCountHeader(byte[] data, out string reason)
        {
            var tag = IndexOf(data, _meshAndBspTag, 0, Math.Min(data.Length, 4096));
            if (tag < 0)
            {
                reason = "no ASCII [MeshAndBsp] entry";
                return null;
            }

            // u32 version, u32 size, then the mesh's chunks: u16 id, u32 length.
            var blob = Array.IndexOf(data, (byte)'\n', tag) + 1;
            var blobSize = BitConverter.ToInt32(data, blob + 4);
            var blobEnd = Math.Min(data.Length, blob + 8 + blobSize);
            var chunk = blob + 8;
            while (chunk + 6 <= blobEnd && BitConverter.ToUInt16(data, chunk) != _meshMaterialsChunk)
                chunk += 6 + BitConverter.ToInt32(data, chunk + 2);
            if (chunk + 6 > blobEnd)
            {
                reason = "no materials chunk";
                return null;
            }

            var chunkLength = BitConverter.ToInt32(data, chunk + 2);
            var body = chunk + 6;
            var bodyEnd = Math.Min(blobEnd, body + chunkLength);
            var objects = IndexOf(data, _objectsTag, body, Math.Min(bodyEnd, body + 256));
            var end = objects < 0 ? -1 : IndexOf(data, _endTag, objects, Math.Min(bodyEnd, objects + 64));
            if (end < 0 || end + _endTag.Length + 4 > bodyEnd)
            {
                reason = "no binary materials archive header";
                return null;
            }

            var count = BitConverter.ToInt32(data, end + _endTag.Length);
            if (data[end + _endTag.Length] != (byte)'\n')
            {
                reason = $"material count {count} isn't affected";
                return null;
            }

            // "END\n\n" -> "END\0": one byte shorter, the chunk and the MeshAndBsp blob shrink with it.
            var fixedData = new byte[data.Length - 1];
            Buffer.BlockCopy(data, 0, fixedData, 0, end + 3);
            fixedData[end + 3] = 0;
            Buffer.BlockCopy(data, end + _endTag.Length, fixedData, end + 4, data.Length - end - _endTag.Length);
            BitConverter.GetBytes(chunkLength - 1).CopyTo(fixedData, chunk + 2);
            BitConverter.GetBytes(blobSize - 1).CopyTo(fixedData, blob + 4);

            reason = $"material count {count} starts with a newline byte";
            return fixedData;
        }

Proposed upstream fix (prepared locally in ZenKit, not submitted yet)

ReadArchiveBinary::read_header consumes exactly END, one line break (\n or \r\n) and at most one empty line; END\0 is accepted too. ArchiveAscii / ArchiveBinsafe are not affected.

  • Unit test (binary header followed by data starting with 0x0A, LF/CRLF/NUL) fails on current ZenKit, passes with the fix; an in-memory world with 266 materials reproduces the bad_alloc and loads with the fix.
  • All 23 vanilla G1/G2/Addon worlds load with identical counts before and after.

Suggested commit: fix(ReadArchiveBinary): only consume the header's line breaks after the second END

Before we go to the ZenKit author

  • Test the patched ZenKit with the real ANCIENTTEMPLE.ZEN (Mroczne Tajemnice) - not done yet, the file wasn't available
  • Rebuild libzenkitcapi (Windows, Quest, Pico) with the fix and load the world in Gothic-Unity
  • Open the ZenKit issue + PR with the results (GothicKit/ZenKit)
Lingua principale
C#
Stelle
33
Fork
1
Merge medio
2g 4h
PR unite (30g)
76

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Gothic-Unity-Project/Gothic-Unity

Tutte le issue di Gothic-Unity-Project/Gothic-Unity

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.