描述
Below is a WIP for mapping out what supporting the torrent protocol will look like for rkt.
discovery
magnet links
Example:
magnet:?xt=urn:btih:{infohash}
&dn={os}-{arch}-{name}-{version}
&tr={URL-escaped tracker URL}">
https://en.wikipedia.org/wiki/Magnet_URI_scheme https://en.wikipedia.org/wiki/Magnet_URI_scheme#Features_and_Clients
Magnet links exist in order to avoid downloading a file containing metadata -- instead the link itself contains the metadata. This is useful in some contexts, but for appc discovery, it is not optimal. A magnet link cannot represent multiple versions of something -- only a single "torrent". This means we would have to implement the discovery URL scheme to find the correct magnet URL to use. At this point we're downloading an HTTP body that contains a magnet link -- which may as well just contain a torrent file. The only difference would be the encoding and the lack of many of BitTorrent's features that do not have standards for being encoded into a magnet link.
Hypothetical discovery HTML:
<meta name="ac-discovery" content="https://storage.example.com/{os}/{arch}/{name}-{version}.{ext}.magnet">
That being said, rkt could easily support loading a magnet link, it just seems suboptimal for discovery.
torrent files
Hypothetical discovery HTML:
<meta name="ac-discovery" content="https://storage.example.com/{os}/{arch}/{name}-{version}.{ext}.torrent">
A neat optimization: check for an HTTPS URL along with the torrent and add that as a web-seed to the torrent data before downloading.
download
client
open questions
Q. How long do we seed?
- Until the download finishes
- In the background (via systemd unit file)
- For how long? (CLI flag)
Q. Where do we download/seed to/from?
- Directly into the CAS
- Into an intermediate file that is then
rkt fetched into the CAS.
libraries
- https://github.com/anacrolix/torrent
- designed to be more modular and library-like
- authored by a Googler
- https://github.com/jackpal/Taipei-Torrent
- fastest
- has been around for a few years
- designed just to produce their binary
- https://github.com/steeve/libtorrent-go
- bindings to the defacto torrent client C library
- swig, yuck
tracker
A tracker is a server used to distribute peers to those interested in a specific file. You can get away with using the BitTorrent DHT, but this can often be slow whereas a tracker is instanteous at returning peers.
libraries
- https://github.com/chihaya/chihaya
- best choice, IMO
- Go
- co-authored by a coreo
- used to distribute containers at facebook
- http://erdgeist.org/arts/software/opentracker/
- need to run multiple instances for ipv4/ipv6 and swarms cannot interact
- C
- has been stable and in prod forever