Support import of partial CARs without prefetching the full DAG

Abierto
#2,380 6 comentarios 1 reacción 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
javascript, react
Área
frontend

Línea de trabajo

Comienza con el flujo de la interfaz web para importar un archivo CAR y el comportamiento de ipfs dag import descrito en el issue; después, sigue el proceso de cómo se fijan los bloques importados antes de la recolección de basura. Se considera completado cuando un usuario puede importar un CAR parcial, conservar mediante la GC los enlaces que contiene sin recuperar previamente el DAG completo y elegir si se conserva contenido más amplio.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

effort/days exp/intermediate help wanted P2 status/blocked

Is your feature request related to a problem? Please describe.

H: As a user, I would expect that when I import a CAR file that it is retained, including all the links it contains, and that parts are not lost through GC.Many links in my NFTs are of this type and as the owner of these I would also like to save them and make them available.
I would be interested to know whether all links are retained when I import the CAR file into Storacha or Filbase.

Describe the solution you'd like

We should allow users the option of pinning all content automatically when importing a car, so NFT owners do not lose content when GC is ran, but they can also only import the content for the car they import.

Describe alternatives you've considered

  • Auto pinning all blocks/CIDs in a car recursively - bad because sometimes folks don't want to do this.. but then they could also remove it easily manually.. which is more likely: importing a car and deleting it later, or importing a car and always keeping all of it?
  • ???

Additional context

From user convo on slack

C: I have downloaded a partial car file of collection_cid/image123.pngWhen I import that car file into kubo with ipfs dag import collection_cid_image123.png.car
Then done ipfs files cp /ipfs/collection_cid/image123.png
How to ensure the parent cid (collection cid) is not garbage collected?

C: I have tested
ipfs pin add <collection_cid>/image.png
ipfs pin add -r=false <collection_cid>
Now I can fetch the ipfs link <collection_cid>/image.png
But when I run garbage collection I cant fetch the link <collection_cid>/image.png anymore. I can only fetch the cid of the image.png
How to prevent, that the parent CID <collection_cid> of the link is not garbage collected?

C: I am offline and I dont want to do ipfs pin add <collection_cid> online, because it then pins the whole collection directory with all files inside which is super big in size.

C: When offline. I import the car and pin the root and parent cid. And the I can fetch the link /image123.png But after running garbage collection the link is gone.

H: ipfs pin add --recursive=false <collection_cid> that will pin only the root cid without pinning the full collection, so that it can resolve <collection_cid>/image.png just fine later. This is all assuming the collection is not huge and the root cid is not a "sharded" repository spanning several blocks.

C: Many thanks Hector. That helps me a lot. 🙏 Yes, that's right, when I direct pin the collection_cid with
ipfs pin add --recursive=false collection_cid
my link <collection_cid>/image123.png is resolved without any problems, even when I run the garbage collection.
But for some very large collections_cids, that's not enough, because of the issue you alluded to in your answer.How should I proceed then? :face_with_monocle:
I don't want the links, which are actually in the car file, to be lost due to garbage collection.
Example:
https://ipfs.io/ipfs/bafybeih4aaint73mwasanappkamkimwjaoe2m7pm6cagfoamjutxzbt76a/7345.png?format=car

C: Now I understand why the links that are present in the CAR files no longer work after I ran the garbage collection. I would not have expected that. 😢 Why is there no option for ipfs dag import to pin the root cid (recursive pin) and also all parent cids (direct pin).
Even the import of car files in ipfs-desktop via the webgui does not do this. Then the car import is useless for me if all links that are present in the car file are lost after running garbage collection. 😢

H: so... if the root directory is sharded, you would need to pin at least one level of subblocks... you can ipfs refs -r --max-depth 1 and then pin each cid individually. That should be enough for moderately big directories already.

C: The complicated process should take place automatically when importing the car file with ipfs dag import carfile.car or in the webui Import CAR file.

cc @lidel @agmap

Lenguaje dominante
JavaScript
Estrellas
1.6k
Forks
539
Métricas de merge de PR
Sin PR fusionados en 30 d

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ipfs/ipfs-webui

Todos los issues de ipfs/ipfs-webui

Issues similares

Más issues de JavaScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.