Get list of validation results for each rule/check?

Open
#272 10 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
data-engineering, testing-qa

Research direction

Start by tracing how the current validation output is assembled and where ValidationResult instances are created or retained. Define an optional structured mapping for each rule or check while preserving the existing textual summary, then verify that callers can retrieve per-rule details for Dagster asset checks.

Written by the indexing model from the issue text.

Description

Hi,

Would it be possible to add support for returning (optionally) a dictionary or similar structure that maps each rule/check to its ValidationResult instance?

We have a use case for integrating dataframely's schema validation with Dagster asset checks. To enable this, we need a way to programmatically access each rule's validation result so we can dynamically map them to corresponding asset checks.

Currently, the validation output may look like:

ValidationError: 4 rules failed validation:
 - 'reasonable_bathroom_to_bedrooom_ratio' failed for 1 rows
 - 'minimum_zip_code_count' failed for 2 rows
 * Column 'zip_code' failed validation for 1 rules:
   - 'min_length' failed for 1 rows
 * Column 'num_bedrooms' failed validation for 1 rules:
   - 'nullability' failed for 2 rows

Hence, it would be great to have an API to obtain validation results in a structured mapping, e.g.:

{
    "reasonable_bathroom_to_bedrooom_ratio": ValidationResult(...),
    "minimum_zip_code_count": ValidationResult(...),
    "zip_code_min_length": ValidationResult(...),
    ...,
}

This would allow us to:

  • Dynamically map each check to a Dagster asset check.
  • Retrieve per-rule validation details programmatically, not just from the textual summary.

Thanks! :)

Dominant language
Python
Stars
618
Forks
21
Avg merge
14h 34m
Merged PRs (30d)
7

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Quantco/dataframely

All issues in Quantco/dataframely

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.