mobidata-bw/postgis-gtfs-importer

investigate possible curl-mirror bug (IPL bwgesamt GTFS import)

Open

#10 建立於 2024年10月29日

在 GitHub 查看
 (5 留言) (0 反應) (0 負責人)JavaScript (1 fork)github user discovery
bughelp wanted

倉庫指標

Star
 (6 star)
PR 合併指標
 (PR 指標待抓取)

描述

In >1 IPL deployment, the following error occurred when mirroring and importing the "bwgesamt" GTFS feed:

+ unzip -d /var/gtfs/gtfs /var/gtfs/gtfs.zip
warning [/var/gtfs/gtfs.zip]: 3428352 extra bytes at beginning or within zipfile
(attempting to process anyway)
file #1: bad zipfile offset (local header sig): 3428352
(attempting to re-compensate)
inflating: /var/gtfs/gtfs/agency.txt
error: not enough memory for bomb detection

A bug in curl-mirror seems likely that

  • either causes parts of new and old versions of the zip file to be "merged" (written into the same file); or
  • doesn't properly continue previously aborted downloads.

However, it could also be that the "bwgesamt" zip file isn't getting updated atomically, in which case a curl-mirror run "caught" the update process right while it had been happening. @NVBWSeifert can you rule out this cause?

貢獻者指南