Evtx.BinaryParser.OverrunBufferException raised unexpectedly

Start with the reported failure path in Evtx/Nodes.py while parsing substitutions and loading templates, then trace the buffer reads in Evtx/BinaryParser.py and reproduce with one of the named Windows 7 EVTX files using evtx_dump.py. The report identifies several affected logs and Python versions; done means those files parse without runaway allocations or buffer overrun exceptions. The issue offers to provide sample files, but no sample or specific failing test is named.

Hi,

I'm seeing exception Evtx.BinaryParser.OverrunBufferException raised unexpectedly when parsing event logs "Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx", "Microsoft-Windows-WindowsUpdateClient%4Operational.evtx", and "Microsoft-Windows-CAPI2%4Operational.evtx" (all extracted from a Windows 7 testing instance) using evtx_dump.py.

For Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx, the same behaviour is seen across Windows with Python 2.7 and Ubuntu with Python 2.7 / 3.5. Traceback from Ubuntu with Python 3.5 as follows:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 273, in unpack_byte
    return struct.unpack_from("<B", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1001, in substitutions
    ofs = self.tag_and_children_length()
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 962, in tag_and_children_length
    for child in self.children():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 952, in children
    return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 153, in _children
    self._chunk, self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 777, in __init__
    parent=self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 398, in add_template
    self._load_templates()
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 380, in _load_templates
    token = self.unpack_byte(ofs - 10)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 275, in unpack_byte
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x240f0ff7, buffer length: 0x101000)

For Microsoft-Windows-WindowsUpdateClient%4Operational.evtx, Python 3.5 on Ubuntu gives Evtx.BinaryParser.OverrunBufferException, while Python 2.7 on Windows / Ubuntu gives a MemoryError. Python 3.5 traceback as follows:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 273, in unpack_byte
    return struct.unpack_from("<B", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1006, in substitutions
    type_ = self.unpack_byte(ofs + 0x2)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 275, in unpack_byte
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x101000, buffer length: 0x101000)

Python 2.7 traceback (from Ubuntu) as follows:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Nodes.py", line 1004, in substitutions
    for _ in range(sub_count):
MemoryError

And similarly for Microsoft-Windows-CAPI2%4Operational.evtx, Python 3.5 on Ubuntu gives Evtx.BinaryParser.OverrunBufferException, while Python 2.7 on Windows / Ubuntu BSODs / has process 'Killed'. Traceback for Python 3.5 on Ubuntu as follows:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 302, in unpack_word
    return struct.unpack_from("<H", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 2 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 191, in render_root_node
    return render_root_node_with_subs(root_node, subs)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 176, in render_root_node_with_subs
    rec(c, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 166, in rec
    sub = render_root_node(sub.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1005, in substitutions
    size = self.unpack_word(ofs)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 304, in unpack_word
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x101000, buffer length: 0x101000)

I can forward the EVTX files in question if that assists?