separate rest client and core scraper library
#47 opened on Mar 8, 2026
Repository metrics
- Stars
- (1 star)
- PR merge metrics
- (PR metrics pending)
Description
currently, both methods of running the project (standalone script, rest client that calls script) rely on each other, but this is not ideal. We want to be able to debug and test the scraper without relying on the backend calling it.
separate both of these modes, so that they can be ran independently of each other
(
script should be called as child_process with it's own process.env, which would require less refactoring but more memory
or
script should be called as an exposed function, maybe wrap as external library that can be imported. this would require more refactoring, but will be more efficient.
)