Mirror released archives to the Internet Archive

Open
  • Is something wrong?
#389 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
csharp
Domain
backend

Research direction

Start by reading RFC 0031 and the content-manager-design materials referenced in the issue; the proposed work affects the watcher, release metadata, licensing policy, and takedowns. The issue explicitly calls for a short RFC before implementation, so first resolve the allowlist, opt-out, account, job, backfill, and takedown questions. Done would mean an agreed design; no implementation files or tests are named.

Written by the indexing model from the issue text.

Description

area:infra area:network

Mirror released archives to the Internet Archive

Priority: low, for the roadmap. Other things come first (Maxi on Discord, 2026-09-21).

Why

A mod that disappears from GitHub or SpaceDock is gone for everyone, even though the index still lists it. A repository gets deleted, an account goes away, SpaceDock has an outage or closes. The stamped release file still has the SHA-256, but nothing to download.

CKAN solves this for KSP by mirroring mods whose license allows it to the Internet Archive. When the original download fails, the client falls back to the copy there.

How it could fit

The index already has the place for it: download.mirrors in a release file (RFC 0031), which the watcher appends to after it has verified the bytes against the stamped sha256. An Internet Archive copy would be one more mirror of that kind.

  1. After a release is stamped, the watcher uploads the archive to an Internet Archive item, one item per listing or per release.
  2. It downloads it back, checks the SHA-256, and appends the archive.org URL to download.mirrors.
  3. Borea already tries the mirrors when the main URL fails, so clients need no change.
Open questions
  • Licenses. Only archives whose license allows redistribution can be uploaded, for example MIT, GPL or CC-BY. LicenseRef-*, all rights reserved and anything unclear stays out. Needs a clear allowlist of SPDX identifiers.
  • Opt out. Should an author be able to say no, even with an open license? Probably yes, with a field in the listing.
  • Account and keys. An Internet Archive account for KSAModding and its upload keys as a secret of the watcher. Who owns the account?
  • Size and time. Uploads cost time on the runner. KSP-Redux is 1.86 GB. It could run as its own job, not in the watcher tick.
  • Backfill. Mirror every release that is already stamped, or only new ones?
  • Takedowns. When a listing is taken down, the Internet Archive copy has to go too. POLICY.md would need a line on it.
  • Design. This probably needs a short RFC in content-manager-design first, because it touches licensing and the index policy.
Not in scope
  • Hosting archives ourselves.
  • Mirroring anything that is not a stamped release.
Dominant language
C#
Stars
29
Forks
3
Avg merge
17h 21m
Merged PRs (30d)
291

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from KSAModding/Borea

All issues in KSAModding/Borea

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.