Support import of partial CARs without prefetching the full DAG

Aperta
#2,380 6 commenti 1 reazione 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
javascript, react
Ambito
frontend

Direzione di ricerca

Inizia con il flusso dell’interfaccia web per importare un file CAR e con il comportamento di ipfs dag import descritto nell’issue, quindi traccia il modo in cui i blocchi importati vengono pinnati prima del garbage collection. Il lavoro è completato quando un utente può importare un CAR parziale, preservare durante il GC i link che contiene senza effettuare il prefetch dell’intero DAG e scegliere se mantenere contenuti più ampi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

effort/days exp/intermediate help wanted P2 status/blocked

Is your feature request related to a problem? Please describe.

H: As a user, I would expect that when I import a CAR file that it is retained, including all the links it contains, and that parts are not lost through GC.Many links in my NFTs are of this type and as the owner of these I would also like to save them and make them available.
I would be interested to know whether all links are retained when I import the CAR file into Storacha or Filbase.

Describe the solution you'd like

We should allow users the option of pinning all content automatically when importing a car, so NFT owners do not lose content when GC is ran, but they can also only import the content for the car they import.

Describe alternatives you've considered

  • Auto pinning all blocks/CIDs in a car recursively - bad because sometimes folks don't want to do this.. but then they could also remove it easily manually.. which is more likely: importing a car and deleting it later, or importing a car and always keeping all of it?
  • ???

Additional context

From user convo on slack

C: I have downloaded a partial car file of collection_cid/image123.pngWhen I import that car file into kubo with ipfs dag import collection_cid_image123.png.car
Then done ipfs files cp /ipfs/collection_cid/image123.png
How to ensure the parent cid (collection cid) is not garbage collected?

C: I have tested
ipfs pin add <collection_cid>/image.png
ipfs pin add -r=false <collection_cid>
Now I can fetch the ipfs link <collection_cid>/image.png
But when I run garbage collection I cant fetch the link <collection_cid>/image.png anymore. I can only fetch the cid of the image.png
How to prevent, that the parent CID <collection_cid> of the link is not garbage collected?

C: I am offline and I dont want to do ipfs pin add <collection_cid> online, because it then pins the whole collection directory with all files inside which is super big in size.

C: When offline. I import the car and pin the root and parent cid. And the I can fetch the link /image123.png But after running garbage collection the link is gone.

H: ipfs pin add --recursive=false <collection_cid> that will pin only the root cid without pinning the full collection, so that it can resolve <collection_cid>/image.png just fine later. This is all assuming the collection is not huge and the root cid is not a "sharded" repository spanning several blocks.

C: Many thanks Hector. That helps me a lot. 🙏 Yes, that's right, when I direct pin the collection_cid with
ipfs pin add --recursive=false collection_cid
my link <collection_cid>/image123.png is resolved without any problems, even when I run the garbage collection.
But for some very large collections_cids, that's not enough, because of the issue you alluded to in your answer.How should I proceed then? :face_with_monocle:
I don't want the links, which are actually in the car file, to be lost due to garbage collection.
Example:
https://ipfs.io/ipfs/bafybeih4aaint73mwasanappkamkimwjaoe2m7pm6cagfoamjutxzbt76a/7345.png?format=car

C: Now I understand why the links that are present in the CAR files no longer work after I ran the garbage collection. I would not have expected that. 😢 Why is there no option for ipfs dag import to pin the root cid (recursive pin) and also all parent cids (direct pin).
Even the import of car files in ipfs-desktop via the webgui does not do this. Then the car import is useless for me if all links that are present in the car file are lost after running garbage collection. 😢

H: so... if the root directory is sharded, you would need to pin at least one level of subblocks... you can ipfs refs -r --max-depth 1 and then pin each cid individually. That should be enough for moderately big directories already.

C: The complicated process should take place automatically when importing the car file with ipfs dag import carfile.car or in the webui Import CAR file.

cc @lidel @agmap

Lingua principale
JavaScript
Stelle
1.6k
Fork
539
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ipfs/ipfs-webui

Tutte le issue di ipfs/ipfs-webui

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.