v2.1.0 JSON-LD context produces wrong or missing RDF triples, Blank Nodes

オープン
#323 コメント 0 件 リアクション 1 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
json, python
領域
data

調査の方向性

Start with oemetadata/v2/v21/context.json, example.json, and schema.json, then reproduce the expansion with rdflib and PyLD as described. Compare the generated RDF against the issue's namespace, term, class/property, prefix, and datatype findings; done means the v2.1 context and example no longer lose or invalidate the specified triples and regression coverage is added.

索引モデルが issue の本文から書いたものです。

説明

status: active 🚧 type: bug 🐛

Description of the issue

When oemetadata/v2/v21/example.json is converted to RDF with oemetadata/v2/v21/context.json, some metadata is silently dropped and several triples use wrong or invalid IRIs.
JSON Schema validation still passes, so none of this is visible without converting to RDF.

Expected: every key in the schema either maps to a well-known predicate with the correct datatype, or is explicitly marked as JSON-only.
Current: the problems below. They are ordered by impact.

1. Wrong namespaces for dct and dcat (regression from v2.0.x)
"dcat": "https://www.w3.org/ns/dcat#",
"dct": "https://purl.org/dc/terms/",

Both namespaces are defined with http://. With https://, every dct:* and dcat:* term (title, description, license, subject, spatial, temporal, keyword, accessURL, …) becomes a different, non-standard IRI.
Standard tools and queries that look for dct:title or dcat:keyword will not find them.
v2/v20/context.json and production still use the correct http:// versions. The change came in with 0d86e74 (#296) and has already been copied to oemetadata/latest/context.json on develop.

2. Keys without a term definition are dropped from the RDF

JSON-LD processors silently drop any key the context does not define. There is no error.
These keys exist in schema.json/example.json but not in context.json:

Key Probable cause What is lost
datasetLicenses Renamed from licenses (#296); context not updated All dataset license information
resourceLicenses Same All resource license information
projectContext New in v2.1, no term definition Project, funding, publisher, contact
mappingRelation Added in #302 without a term definition Every skos:exactMatch etc. on isAbout / valueReference
3. "@id": null in the example is rejected by strict processors

example.json contains "@id": null six times (in the empty valueReference entries).
pyld fails the whole document with Invalid JSON-LD syntax; "@id" value must be a string.
Some tools (e.g. rdflib) accept it, so whether the file parses depends on the tool.

4. Classes used as properties

These terms map to an RDF class, but they are used as a property (predicate):

Key Mapped to Suggested property
contributors foaf:Agent dct:contributor (or prov:qualifiedAttribution, see below)
resources dcat:Dataset dcat:distribution (or dct:hasPart)
resource (foreign key reference) dcat:Distribution a csvw reference property, e.g. csvw:resource
timeseries dct:PeriodOfTime dct:temporal, with @type dct:PeriodOfTime on the object

A class used as a predicate is valid RDF syntax, but the result has no useful meaning and does not work with DCAT/Dublin Core tooling.

5. Broken ncit prefix
"ncit": "http://purl.obolibrary.org/obo/ncit.owl",

The prefix has no trailing separator, so nullable → ncit:NCIT_C47840 expands to http://purl.obolibrary.org/obo/ncit.owlNCIT_C47840, which is not a valid term IRI.
The correct IRI is http://purl.obolibrary.org/obo/NCIT_C47840. That means prefix http://purl.obolibrary.org/obo/, as already used for IAO.
This prefix is the same in v2.0.

6. Datatypes that don't match the values
Key Declared type Example value
embargoPeriod.start / .end xsd:dateTime "2024-10-11" (a date)
sources.publicationYear xsd:date "2023" (a year → xsd:gYear)

The resulting literals are ill-typed and are rejected or ignored by validating RDF stores and SHACL.

Steps to Reproduce

  1. pip install rdflib pyld
  2. Merge oemetadata/v2/v21/example.json with the @context from oemetadata/v2/v21/context.json. Replace the example's remote @context URL, and remove the "@id": null entries so the document parses at all.
  3. Parse with rdflib (Graph().parse(data=..., format="json-ld")) and inspect the predicates.
    Alternatively paste the merged document into the JSON-LD Playground and look at the N-Quads tab.
  4. You will see:
    • https://purl.org/dc/terms/… and https://www.w3.org/ns/dcat#… predicates;
    • no license, project context or mappingRelation triples;
    • …/ncit.owlNCIT_C47840;
    • rdflib warnings for the ill-typed date literals.
  5. Running pyld.jsonld.expand() on the unmodified merged document fails on "@id": null.

Ideas of solution

  1. Namespaces: set dct and dcat back to http://, in both v21/context.json and latest/context.json. This is a trivial fix with no downside.

  2. Missing terms: add term definitions for datasetLicenses, resourceLicenses (e.g. dct:license, @container: @set) and projectContext (e.g. prov:wasGeneratedBy with a scoped context for its sub-keys).
    mappingRelation needs a modelling decision. Its value (skos:exactMatch) really describes which predicate should connect the column to the concept, and a plain term definition can't express that. The options are:

    • (a) map it as a plain literal / @vocab IRI for now;
    • (b) model the mapping as its own node, e.g. SSSOM-style subject / predicate / object;
    • (c) have a generator emit column skos:exactMatch concept directly.

    (a) is quick. (b) or (c) produce usable RDF.

  3. "@id": null: omit @id in the example/template when there is no value, instead of writing null. Alternatively, have the template use an empty object and document that keys without a value should be left out.

  4. Classes as properties: remap to the properties suggested above. For contributors, use dct:contributor for now. The proper model separates the contribution (role, date, object; a qualified attribution) from the agent (person, with an IRI such as an ORCID). That could be done in a follow-up.

  5. ncit: change the prefix to http://purl.obolibrary.org/obo/ (or map nullable to the full IRI). Consider backporting to v2.0.

  6. Datatypes: use xsd:date for the embargo period, or change the example to full datetimes. Use xsd:gYear for publicationYear.

  7. Prevent regressions: add a test that expands the example with the context and fails when a schema key is neither mapped nor on an explicit "JSON-only" allow-list, when a prefix IRI does not end in / or #, or when a literal is ill-typed. That test would have caught items 1, 2, 5 and 6.

Out of scope here and worth a separate enhancement issue: many nodes end up as blank nodes (licenses, contributors, sources, columns, …) because they have no @id. This makes them impossible to link across datasets.

Context and Environment

  • Version used: v2.1.0 (in development, develop / feature-298-module-energy-system)
  • Operating system: Linux (WSL2)
  • Environment setup and (python) version: Python 3.10.12, rdflib 7.6.0, PyLD 3.3.0; also checked with the JSON-LD Playground

Workflow checklist

主要言語
Python
スター
26
フォーク
4
PR マージ指標
30日以内にマージされた PR はありません

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

OpenEnergyPlatform/oemetadata のほかの issue

OpenEnergyPlatform/oemetadata の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。