documentationgood first issue
Metriche repository
- Star
- (4 stelle)
- Metriche merge PR
- (Metriche PR in attesa)
Descrizione
There is a recurring but intermittent error that appears when ingesting a shapefile into R from the database. This error seems to be somehow related to connect/disconnect pattern, as it occurs more frequently if connections are not well managed. However, it does appear after multiple consecutive spatial table imports.
More context:
- spatial table imports are done in batches of 1000 rows at a time. This was initially put in place to avoid memory problems from spatial imports of ~3000 rows.
- even with the batch import, mutliple consecutive imports (pulling the same 3k row table in 3-4 times) will cause the error.
- Since the problem is still occurring after a total of ~10000 rows are imported during a single session, it is worrisome to imagine a larger table failing to load even once.
- The problem always goes away/ memory resets when the R session is restarted. Obviously, any imported data is cleared and all packages are unloaded.