derhuerst/hafas-generate-gtfs

persistent queue

Open

#1 aberto em 21 de mar. de 2020

Ver no GitHub
 (6 comments) (0 reactions) (2 assignees)JavaScript (0 forks)github user discovery
enhancementhelp wanted

Métricas do repositório

Stars
 (6 stars)
Métricas de merge de PR
 (Nenhuma PRs mesclada em 30d)

Description

A persisting queue will allow the script to fail and/or be cancelled, and resumed later. It will also make hafas-generate-gtfs work in memory-constrained environments, or with very large areas.

Changes necessary:

  1. Specify all tasks (departures fetching, trips fetching) in a JSON-serialisable format. (['fetch-trip', tripId, lineName], etc).
  2. Implement an in-memory store for stops, lines & trips (see API below).
  3. Switch from queue to a persisting one. (I'd like to try bullmq.) Also mention in the readme that Redis is required.

store API

{
	hasStop: async id => Boolean,
	getStop: async id => Stop,
	putStop: async (id, stop) => {},
	// same with lines & trips
}

Guia do colaborador