Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

BUG: Improve Package from PurlDB failures

Abierto
  • ¿Algo no está bien?
#303 6 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@tdruez ya está trabajando en esto.

Desde el 5/5/2025.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

bug HighPriority PackageSet

Extracted from https://github.com/aboutcode-org/dejacode/issues/295#issuecomment-2824782627

Running "Improve Package from PurlDB" fails with duplicate key value violates unique constraint"component_catalog_packag_dataspace_id_type_namesp_c6620419_uniq"DETAIL:Key(dataspace_id,type,namespace,name,version,qualifiers,subpath,download_url,filename)=(3,npm,,parse-json,4.0.0,,,https://registry.npmjs.org/parse-json/-/parse-json-4.0.0.tgz,parse-json-4.0.0.tgz)alreadyexists. since assigning the download_url would make it a fully duplicate package.

Attempting to enhance the package in the product with data from PurlDB fails, because assigning the download URL would violate the uniqueness constraint that covers dataspace_id, type, namespace, name, version, qualifiers, subpath, download_url, and filename

This is a corner case where the data pulled from the PurlDB can trigger a "unique constraint" violation when applied to a package.

Context

PURL from a CycloneDX SBOM: pkg:npm/[email protected]
Note that in that data source context, no download_url is provided.

"There are multiple entries in the PurlDB for this Package." -> 4 results in the PurlDB for this PURL.

https://public.purldb.io/api/packages/?purl=pkg:npm/[email protected]

[
        {
            "uuid": "d1eb90f6-5115-49b8-a5f8-782e948cbd3d",
            "filename": "parse-json-4.0.0.tgz",
            "purl": "pkg:npm/[email protected]",
            "type": "npm",
            "name": "parse-json",
            "version": "4.0.0",
            "download_url": "https://registry.npmjs.org/parse-json/-/parse-json-4.0.0.tgz",
            "sha1": "be35f5425be1f7f6c747184f98a788cb99477ee0",
            "package_content": null,
            "package_sets": []
        },
        {
            "uuid": "bd3c0519-50d6-47a9-b01a-d3241e0ef641",
            "filename": null,
            "package_sets": [],
            "purl": "pkg:npm/[email protected]",
            "type": "npm",
            "namespace": "",
            "name": "parse-json",
            "version": "4.0.0",
            "download_url": "https://registry.npmjs.com/parse-json/-/parse-json-4.0.0.tgz",
            "sha1": "be35f5425be1f7f6c747184f98a788cb99477ee0",
            "package_sets": []
        },
        {
            "uuid": "0ff3a534-5570-4289-8560-6e46bb3bad4f",
            "filename": "parse-json-4.0.0.tgz",
            "purl": "pkg:npm/%40types/[email protected]",
            "type": "npm",
            "namespace": "@types",
            "name": "parse-json",
            "version": "4.0.0",
            "download_url": "https://registry.npmjs.org/@types/parse-json/-/parse-json-4.0.0.tgz",
            "package_content": null,
            "package_sets": [
                {
                    "uuid": "721da624-9d02-401f-82b6-46142449635d",
                    "packages": [
                        "https://public.purldb.io/api/packages/0ff3a534-5570-4289-8560-6e46bb3bad4f/",
                        "https://public.purldb.io/api/packages/ec74c5e3-676b-489a-a059-4176bdfe6ea8/"
                    ]
                }
            ]
        },
        {
            "uuid": "680f4c25-68b3-4e21-8c16-6db2a822e752",
            "filename": null,
            "purl": "pkg:npm/%40types/[email protected]",
            "type": "npm",
            "namespace": "@types",
            "name": "parse-json",
            "version": "4.0.0",
            "download_url": "https://registry.npmjs.com/@types/parse-json/-/parse-json-4.0.0.tgz",
            "package_content": null,
            "package_sets": [
                {
                    "uuid": "c4608309-c909-4a86-bf75-c3363d50bf98",
                    "packages": [
                        "https://public.purldb.io/api/packages/680f4c25-68b3-4e21-8c16-6db2a822e752/",
                        "https://public.purldb.io/api/packages/ec74c5e3-676b-489a-a059-4176bdfe6ea8/"
                    ]
                }
            ],
        }
    ]
  1. Since the provided purl has no namespace, ie: pkg:npm/[email protected], the entries for pkg:npm/%40types/[email protected] should not be returned
  2. The default ordering of the results should be by each PURL field. From packageurl.contrib.django.models.PackageURLQuerySetMixin.order_by_package_url:
PACKAGE_URL_FIELDS = ("type", "namespace", "name", "version", "qualifiers", "subpath")
def order_by_package_url(self):
    """Order by Package URL fields."""
    return self.order_by(*PACKAGE_URL_FIELDS)
  1. The 2 entries for pkg:npm/[email protected] are the same, the sha1 id equal, the only difference is the domain in the download_url: npmjs.org vs npmjs.com. The issue in the DejaCode context using the PurlDB is which one of those package should we use?
    An approach could be to take the common fields values from the multiple entries. In this case, as the only know value on the DejaCode side is pkg:npm/[email protected], we could ignore the download_url and hash and import everything that is shared across the PurlDB records.

TODO (Fix) in DejaCode

  1. The task should not break, any issues happening during the "Improve Packages from PurlDB" should be properly handled and logged during the improve_packages_from_purldb task.

  2. Using data pulled from PurlDB onto a package should not raise a unique constraint violation.

Lenguaje dominante
Python
Estrellas
52
Forks
30
Merge medio
4 h 57 min
PR fusionados (30 d)
21

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de aboutcode-org/dejacode

Todos los issues de aboutcode-org/dejacode

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.