openai_harmony.HarmonyError: unexpected tokens remaining in message header

Start with the parse_messages_from_completion_tokens entry point referenced in the traceback and reproduce the failure through the vLLM direct-sampling example linked in the report. Determine the expected behavior when generation ends at the maximum length, then verify the parser handles that case with regression coverage; the issue is complete when the reported HarmonyError no longer occurs for the reproduction.

I encountered the following problem while using vllm (https://cookbook.openai.com/articles/gpt-oss/run-vllm#using-vllm-for-direct-sampling).
It appears that the text has been truncated beyond the maximum length during generation, resulting in it being unparseable.

Traceback (most recent call last):                                                                                                                                           
  File "/mnt/public/kyzhang/MARTI/tests/run_eval_aime.py", line 66, in <module>                                                                                              
    entries = encoding.parse_messages_from_completion_tokens(output_tokens, Role.ASSISTANT)                                                                                  
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                  
  File "/root/miniconda3/envs/gpt-oss/lib/python3.12/site-packages/openai_harmony/__init__.py", line 525, in parse_messages_from_completion_tokens                           
    raw_json: str = self._inner.parse_messages_from_completion_tokens(                                                                                                       
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                       
openai_harmony.HarmonyError: unexpected tokens remaining in message header: ["The", "task:", "Count", "rectangles", "with", "sides", "along", "edges", "or", "diagonals", "of
", "regular", "dodecagon.", "Vertices", "of", "rectangle", "must", "be", "among", "12", "vertices?", "Not", "necessarily,", "sides", "lie", "along", "sides", "or", "diagonal
s,", "so", "rectangle's", "sides", "may", "be", "segments", "connecting", "two", "vertices", "that", "are", "either", "adjacent", "or", "any", "other", "pair", "forming", "a
", "segment", "(diagonal).", "The", "rectangle's", "vertices", "are", "intersection", "points", "of", "such", "segments?", "Wait", "rectangle", "sides", "along", "sides", "o
r", "diagonals:", "meaning", "each", "side", "of", "rectangle", "coincides", "with", "a", "chord", "connecting", "two", "polygon", "vertices", "(either", "side", "or", "diag
onal).", "So", "rectangle", "vertices", "are", "among", "the", "12", "vertices", "of", "polygon.", "Because", "sides", "are", "subsets", "of", "sides", "or", "diagonals", "s
o", "endpoints", "at", "vertices.", "So", "rectangle", "is", "a", "sub-graph", "of", "complete", "graph", "on", "12", "vertices", "with", "edges", "being", "sides", "or", "d
iagonals,", "forming", "a", "rectangle", "(4-cycle", "with", "right", "angles", "in", "Euclidean", "geometry).", "Need", "to", "count", "all", "rectangles", "whose", "vertic
es", "among", "12", "vertices", "and", "whose", "edges", "are", "chords", "of", "the", "regular", "dodecagon.", "We", "need", "to", "find", "all", "quadrilaterals", "with", 
"vertices", "among", "12", "that", "are", "rectangles", "in", "the", "plane", "of", "the", "regular", "dodecagon", "(regular", "12-gon).", "In", "regular", "polygon,", "rect
angle", "can", "be", "symmetric.", "In", "a", "regular", "polygon", "of", "even", "number,", "inscribed", "rectangles", "correspond", "to", "selecting", "two", "opposite", "
vertices", "(16?).", "Let's", "analyze:", "In", "a", "circle,", "any", "rectangle", "inscribed", "must", "have", "opposite", "vertices", "opposite", "on", "circle?", "Not:",
 "Inscribed", "rectangle", "in", "circle:", "its", "vertices", "lie", "on", "circle;", "rectangle", "sides", "chords;", "Opposite", "vertices", "are", "diametrically", "oppo
site?", "For", "any", "inscribed", "rectangle,", "it's", "cyclic", "where", "opposite", "angles", "are", "90", "degrees", "=>", "arcs", "between", "adjacent", "vertices", "a
re", "180-?", "Wait", "property:", "In", "a", "circle,", "a", "rectangle", "is", "a", "cyclic", "quadrilateral", "with", "opposite", "angles", "90°,", "so", "each", "side", 
"chord", "corresponds", "to", "arc", "180°?", "Actually", "rectangle:", "opposite", "angles", "are", "90°,", "so", "arcs", "subtended", "by", 
"opposite", "sides", "sum", "torilateral", "with", "right", "angles", "at", "all", "vertices", "is", "a", "rectangle", "only", "if", "it's", "a", "cyclic", 
"rectangle.", "For", "a", "rectangle", "inscribed", "in", "a", "circle,", "its", "diagonals", "are", "diameters", "of", "the", "circle?", "In", "general,", 
"a", "rectangle", "inscribed", "in", "a", "circle", "must", "have", "its", "diagonals", "be", "diameters?", "Wait:", "In", "a", "rectangle,", 
"diagonals", "are", "equal", "but", "not", "necessarily", "diameters.", "For", "rectangle", "inscribed", "in", "a", "circle,", "the", "center", "of", 
"circle", "is", "intersection", "of", "diagonals.", "So", "diagonal", "length", "=", "diameter?", "Actually", "the", "rectangle's", "circumcircle", 
"passes", "through", "its", "vertices,", "center", "at", "intersection.", "For", "rectangle,", "intersection", "of", "diagonals", "at", "center", "leads", 
"that", "diagonals", "equal", "in", "length", "=", "diameter", "of", "circle.", "Indeed", "for", "a", "rectangle", "inscribed", "in", "circle,", "the", 
"diagonals", "serve", "as", "diameters:", "because", "they", "both", "go", "through", "center,", "equal", "to", "diameter.", "So", "given", "circle,", 
"rectangle", "vertices", "must", "be", "pairs", "of", "opposite", "points", "of", "circle", "coordinates", "such", "that", "XOR?", "But", "not", 
"necessarily", "that", "each", "vertex", "is", "exactly", "diameter", "endpoints.", "The", "rectangle", "requires", "that", "the", "four", "vertices", 
"are", "arranged", "such", "that", "opposite", "vertices", "are", "diametrically", "opposite.", "Let's", "test:", "In", "a", "square", "inscribed", "in", 
"circle,", "vertices", "are", "90°", "apart;", "opposite", "vertices", "180°.", "For", "rectangle", "that", "is", "not", "square,", "still", "opposite", 
"vertices", "are", "180°,", "so", "rectangle", "vertices", "come", "in", "two", "antipodal", "pairs.", "So", "polygon", "vertices", "must", "be", 
"arranged", "in", "such", "that", "we", "choose", "two", "pairs", "of", "opposite", "vertices", "in", "the", "regular", "dodecagon", "that", "are", 
"each", "separated", "by", "6", "steps", "(180°).", "But", "we", "also", "need", "adjacent", "vertices", "of", "rectangle", "correspond", "to", 
"chords", "that", "may", "not", "be", "sides", "or", "long", "diagonals?", "But", "any", "chord", "between", "two", "vertices", "qualifies.", "So", 
"rectangle", "determined", "by", "choosing", "two", "diameters:", "pick", "an", "ordered", "pair", "of", "diameters", "such", "that", "the", 
"quadrilateral", "connecting", "these", "four", "vertices", "in", "order", "yields", "a", "rectangle.", "Given", "a", "12-gon,", "diameters", 
"correspond", "to", "pairs", "of", "opposite", "vertices", "(difference", "6).", "There", "are", "6", "such", "diameters.", "Choose", "any", "two", 
"distinct", "diameters", "that", "are", "not", "parallel?", "But", "rectangle", "sides", "come", "from", "chords", "between", "the", "chosen", 
"vertices.", "Let's", "list", "vertices:", "number", "them", "0..11", "around", "circle.", "Opposite", "pairs:", "(0,6),", "(1,7),", "(2,8),", "(3,9),", "
(4,10),", "(5,11).", "Choose", "two", "diameters:", "say", "(0,6)", "and", "(1,7).", "Connect", "vertices", "in", "sequence", "0,1,6,7?", "But", 
"forming", "rectangle?", "Let's", "order", "them", "properly", "around", "circle:", "we", "need", "quadrilateral", "with", "vertices", "alternating", 
"between", "the", "two", "diameters.", "For", "diameters", "a-b", "and", "c-d", "with", "a", "opposite", "b,", "c", "opposite", "d.", "If", "we", 
"take", "vertices", "a,", "c,", "b,", "d", "in", "that", "order,", "we", "get", "crossing?", "Actually", "quadrilateral", "with", "vertices", "a,c,b,d", 
"around", "circle", "gives", "a", "crossing", "shape", "-", "not", "simple?", "Let's", "think:", "The", "rectangle", "must", "be", "non-self-
intersecting.", "The", "rectangle's", "vertices", "arranged", "such", "that", "sides", "connect", "a", "to", "c,", "c", "to", "b,", "b", "to", "d,", "d", 
"to", "a.", "So", "the", "quadrilateral", "uses", "diagonals", "across", "circle?", "But", "side", "a-c", "is", "chord", "connecting", "a", "to", "c", "
(not", "necessarily", "diameter).", "Similarly", "c-b", "etc.", "But", "does", "this", "produce", "rectangle?", "Since", "a", "and", "b", "are", 
"opposite,", "difference", "6.", "Similarly", "c", "and", "d", "opposite", "difference", "6.", "The", "quadrilateral", "a-c-b-d", "forms", "a", 
"rectangle", "when", "the", "angle", "between", "chords", "a-c", "and", "c-b", "is", "90", "degrees,", "etc.", "We", "can", "brute", "logic:", "For", 
"regular", "12-gon,", "rectangle", "inscribed", "must", "have", "vertices", "symmetric", "about", "center:", "pairs", "opposite.", "That", "yields", 
"all", "rectangles?", "There", "might", "be", "also", "rectangles", "whose", "vertices", "not", "all", "diametrically", "opposite", "but", "still", 
"rectangles?", "Suppose", "vertices", "0,2,6,8?", "Are", "opposite?", "0-6", "opposite", "yes,", "2-8", "opposite", "yes.", "This", "is", "same", 
"pattern", "as", "above.", "So", "any", "selection", "of", "two", "distinct", "diameter", "pairs", "yields", "a", "rectangle?", "Check", 
"coordinates:", "coordinates", "at", "12", "equally", "spaced", "points", "on", "unit", "circle.", "Let", "a", "angle", "0°,", "b", "180°,", "c", "30°,", 
"d", "210°.", "Actually", "30°", "is", "30°,", "210°", "is", "210°,", "difference", "180.", "Quadrilateral", "0°,30°,180°,210°?", "coordinates?", "Does", 
"that", "shape", "have", "right", "angles?", "Compute", "slopes?", "Might", "produce", "rectangle.", "I", "think", "any", "pair", "of", "diameters", 
"gives", "rectangle.", "Number", "of", "ways", "to", "choose", "2", "diameters", "from", "6", "is", "C(6,2)=15.", "For", "each", "pair,", "but", 
"ordering", "of", "vertices", "maybe", "gives", "same", "rectangle?", "Does", "pair", "of", "diameters", "define", "a", "unique", "rectangle", 
"independent", "of", "which", "side", "chosen?", "The", "rectangle", "vertices", "are", "the", "four", "points.", "Reordering", "doesn't", 
"change", "rectangle.", "So", "number", "rectangles", "=", "15.", "But", "also", "consider", "rectangles", "with", "sides", "along", "polygon", 
"sides", "or", "diagonals", "but", "not", "necessarily", "using", "diameters?", "Is", "it", "possible", "that", "rectangle", "vertices", "are", "not", 
"diametric", "pairs?", "Example:", "consider", "rectangle", "with", "vertices", "0,1,6,7:", "0-1", "are", "adjacent,", "6-7", "also", "adjacent,", "0-
6", "diametric,", "1-7", "diametric.", "This", "still", "two", "diameters", "(0-6,1-7).", "So", "all", "rectangles", "indeed", "come", "from", "two", 
"diameters.", "Thus", "answer", "15?", "Double-check", "if", "any", "other", "combinations", "produce", "rectangle", "but", "not", "using", 
"diameters?", "Suppose", "choose", "vertices", "0,3,6,9:", "3-9", "diff", "6", "diam.", "0-6", "diff6,", "yes.", "So", "again", "diameters.", "So", 
"seems", "all", "rectangles", "use", "diametric", "pairs.", "Thus"]