Prometheus reads all .db files from PROMETHEUS_MULTIPROC_DIR without regard

從 prometheus_client/multiprocess.py 中的 MultiProcessCollector.collect 開始，在重現回報的 prometheus-client 0.15.0 traceback 的同時，追蹤它對 merge 和 mmap_dict.py 的呼叫。確定應如何辨識 PROMETHEUS_MULTIPROC_DIR 中的檔案，然後驗證無關或格式錯誤的 .db 檔案不再導致收集程序讀取無效資料，同時有效的 metric 檔案仍能正常運作。

prometheus-client==0.15.0

The method collect on prometheus_client.multiprocess.MultiProcessCollector reads all .db files, leading to memory issues, as it reads SQLite databases.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/........./........../.............py", line 20, in .........................
    data = generate_latest(registry)
  File "/usr/lib/python3.8/site-packages/prometheus_client/exposition.py", line 198, in generate_latest
    for metric in registry.collect():
  File "/usr/lib/python3.8/site-packages/prometheus_client/registry.py", line 97, in collect
    yield from collector.collect()
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 153, in collect
    return self.merge(files, accumulate=True)
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 45, in merge
    metrics = MultiProcessCollector._read_metrics(files)
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 73, in _read_metrics
    for key, value, _ in file_values:
  File "/usr/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 43, in _read_all_values
    value = _unpack_double(data, pos)[0]
struct.error: unpack_from requires a buffer of at least 1634562696 bytes for unpacking 8 bytes at offset 1634562688 (actual buffer size is 12288)

The issue is on collect() method:

def collect(self):
    files = glob.glob(os.path.join(self._path, '*.db'))
    return self.merge(files, accumulate=True)

I believe that there should be some kind of file validation to assure that the file is related to a prometheus metric, e.g:

import re

POSSIBLE_PROMETHEUS_FILENAMES_INCLUDE = {'counter', '...', 'histogram', 'gauge_livesum'}

def collect(self):
    files = glob.glob(os.path.join(self._path, '*.db'))
    #
    # assert files
    valid_files =  [f for f in files if re.split('_\d+.db', f)[0] in POSSIBLE_PROMETHEUS_FILENAMES_INCLUDE]
    #
    #
    return self.merge(valid_files, accumulate=True)