Refresh changed/deleted sources with resumable bounded jobs

Open
  • Is something wrong?
#41 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
38/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
javascript, node.js
Domain
backend, databases, security

Research direction

Read AGENTS.md and docs/PROJECT_CONTEXT.md, then inspect the cache/index adapters and src/index.js alongside the contracts from #38 and #39. Discuss the design with the maintainer before implementation. Done means resumable bounded jobs handle interruptions, source changes and deletions, revoked enrollment, concurrent updates, account switching and restart, with offline tests verifying no duplicate chunks and safe progress and error output.

Written by the indexing model from the issue text.

Description

area:backend area:docs priority:medium status:blocked type:feature

Part of #28 · Target: Durability / v0.6

Goal and scope

Add incremental refresh/status/cancel/resume operations with bounded work, interruption recovery and changed/deleted/denied-source handling. Invalidate outdated derived content without leaking it across accounts. Report checked/stale/failed scope.

Implementation starting points

cache/index adapters, new refresh job state, src/index.js. Follow AGENTS.md and docs/PROJECT_CONTEXT.md; reuse existing authorization, safe-error and synthetic-fixture patterns. New runtime dependencies need a documented reason.

Dependencies and readiness

Depends on #38, #39. Discuss the design before starting implementation; dependent contracts must land first.

Out of scope

No background polling/notifications by default, unbounded full crawl or persisted sessions.

Acceptance criteria and test plan

Tests simulate interrupted jobs, source replacement/deletion, revoked enrollment, stale data, concurrent updates, account switching and restart. Resumption avoids duplicate chunks and progress/errors contain counts and safe codes rather than private content.

Run the relevant offline tests, npm run check and git diff --check; runtime changes also require npm test and npm audit. Report checks not run. No faculty credentials or private course material are required for a contribution.

Contributing

Comment here before starting so the maintainer can coordinate the approach and avoid duplicate work. Fork the repository, branch from master, and open a focused PR referencing this issue. No assignee or delivery deadline is set; required checks and review apply.

Dominant language
JavaScript
Stars
0
Forks
0
Avg merge
1h 25m
Merged PRs (30d)
13

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mahmutmft/courses-mcp

All issues in mahmutmft/courses-mcp

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.