Multiaddr Based Content Routing

未关闭
  • 哪里不对？
#11 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
25/100
Issue 类型
功能
描述清晰度
需要澄清
活跃度
停滞
技术栈
aws
领域
cloud, distributed-systems, networking

调研方向

未指定文件或测试。首先审查 Exchange (Bitswap) 和 Content Routing 的流程，包括如何处理 PeerInfo 和 DHT provide records。完成标准是针对从 non-libp2p multiaddrs 获取数据达成一致的设计并完成实现，同时解决其验证和多块相关的影响。

由索引模型根据 Issue 内容生成。

描述

Context

A standard IPFS data request causes the Exchange (i.e. Bitswap) to search the Content Routing system for a set of PeerInfo objects (which are just PeerIDs + their multiaddrs). The Exchange then takes these PeerInfo objects and requests data from the peers.

This causes us to need libp2p peers to proxy all data available to the Content Routing system, but if the data is available elsewhere shouldn't we be able to access it?

Proposal

I would like to be able to request data from multiaddrs that do not correspond to libp2p peers. For example, if we want to store data with some cloud storage provider like AWS S3 we could put a provide record in the DHT that Hash(Data) lives at /http/mybucket.s3.amazonaws.com/Data.

Motivation

While we could also run an a compute node, like EC2, with a set of IPFS cluster daemons on them with an S3-backed datastore it's certainly more costly. This is even more interesting if we can "draft" data that's publicly available over HTTP into IPFS.

Implications for future work

While the first iteration of this idea is conceptually fairly simple, it has implications for some of our ongoing endeavors. For instance, if we have 1000 small blocks hosted on /http/mysite.com/Data1-1000 that are all part of a single IPLD object we wouldn't be able to just provide the root IPLD node since there's no peer that will be able to tell us where the other 999 blocks are. There are various ways we could extend the protocol to allow us to tell retrievers where the other 999 blocks are, but it's not as simple as with the existing peer based retrieval.

Additionally, we would likely face increasing demand to support large files that are available over HTTP. Since we don't want users to download a lot of data before it's verified we'd probably want to extend the protocol with some ability for the peers advertising the content in the DHT to add (references to) hashes of chunks of the large file that could be verified. Similarly, we'd want to add the ability to download ranges of bytes when presented with a multiaddr that supports that functionality.

I think implementing this functionality could make running "pinning" services much easier and less expensive as well as greatly increasing the amount of content accessible via IPFS. But what about you @stebalien @raulk @bigs ?

主要语言
没有语言数据
星标
38
派生
2
30 天内没有已合并 PR

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

libp2p/notes 的其他 Issue

查看 libp2p/notes 的全部 Issue

相似的 Issue

更多 Cloud Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。