Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Test failures cannot distinguish advisory SHACL results (sh:Warning / sh:Info) from sh:Violation

Aperta
  • Qualcosa non va?
#88 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
testing-qa

Direzione di ricerca

Start in ogc/bblocks/validation/rdf.py at shacl_validate and trace how the validator options and ValidationReportEntry are exposed through the CLI or bblocks-config.yaml. Verify the advisory-severity behavior against the reported pySHACL cases, while preserving warnings in reports and generated docs. Clarify whether the empty uplift output behavior is part of this issue before considering that work done.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Test failures cannot distinguish advisory SHACL results from violations

Summary

bblocks-postprocess marks a building block's test as failed whenever pySHACL
reports conforms == False. pySHACL sets that for a result of any severity,
so a block that emits only sh:Warning or sh:Info is reported identically to
one that emits sh:Violation.

The effect on a register that uses advisory shapes deliberately is that it can
never be green, and "the build is red" stops carrying information.

Evidence from a real register

In the CDIF register (93 blocks), the postprocessor reports 22 failures. Across
every failing item:

severity results
sh:Warning 134
sh:Info 31
sh:Violation 0

Not one violation in the register. Blocks are failed for emitting advice they
were written to emit, e.g.

for cdi InstanceVariable a physical data type should be provided.

20 of the 22 failures are advisory-only. (Of the remaining two, one was a genuine
bug in our own SHACL, now fixed; the other is item 2 below.)

Where it happens

ogc/bblocks/validation/rdf.py:

def shacl_validate(g, s, ont_graph=None):
    validator = pyshacl.Validator(g, shacl_graph=s, ont_graph=ont_graph, options={
        'advanced': True
    })
    ...
    conforms, shacl_result, shacl_report = validator.run()

and then

if not shacl_conforms:
    shacl_errors_found = True
report.add_entry(ValidationReportEntry(
    ...
    is_error=not shacl_conforms,

There is no severity filter, and no CLI option or bblocks-config.yaml key
exposes one.

Proposed fix

pySHACL already supports this natively via allow_warnings, which excludes
sh:Warning and sh:Info from the conforms verdict while still reporting
them. Verified against one of the failing cases above:

options={'advanced': True}                           -> conforms=False  results={'Warning': 1}
options={'advanced': True, 'allow_warnings': True}   -> conforms=True   results={'Warning': 1}

So the change is to pass the option through and expose it, e.g.
--shacl-allow-warnings (and/or a shacl: allow-warnings: true key in
bblocks-config.yaml), defaulting to the current behaviour so nothing changes
for existing registers.

The warnings would still appear in the report and the generated docs - only the
pass/fail verdict changes.

Related: empty uplift output is also a hard failure

A second case with the same shape. An example that is a legitimate bare
{"@id": "..."} reference cannot produce any triples - that is what a node
object with no properties means in JSON-LD - but the uplift step reports

Empty output Turtle example_1_1.ttl created

as an error, failing the block. In our register this is the documented example
for "SKOS concept by URI reference", pointing at a published NERC vocabulary
term; it is correct, and deleting it to get a green build would remove the
documentation of the most common usage of that block.

Suggest this be a warning rather than an error, or suppressed when the source
document is a bare reference.

Why it matters

Both cases fail the same way: a check that cannot tell "broken" from "benign".
In our register the practical consequence was that a genuinely broken composite
profile - a rules.shacl referencing three shapes it did not define, which made
pySHACL reject the file and caused all five of that block's items to fail,
including its requireFail negative tests - sat unnoticed among 21 failures that
did not matter.

With allow_warnings plus that one real fix, the register would go from 22
failing blocks to 1.

🤖 Drafted with Claude Code, reviewed by SMR

Lingua principale
Python
Stelle
1
Fork
6
Merge medio
1m
PR unite (30g)
1

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di opengeospatial/bblocks-postprocess

Tutte le issue di opengeospatial/bblocks-postprocess

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.