Promote ExpectMulticolumnValuesToBeEqual to a supported core Expectation
#12,002 opened on 2026/07/24
Repository metrics
- Stars
- (9,116 個のスター)
- PR merge metrics
- (30d に merged PR はありません)
説明
Use case
I need to assert that three or more columns hold the same value on every row — e.g. a set of amount columns that should always agree, or a value replicated across columns during a join that must stay in sync. Core ships ExpectColumnValuesToBeEqual's two-column form (ExpectColumnPairValuesToBeEqual), but there is no supported N-column generalization. This was raised in discussion #4276 (the standing "suggest an Expectation" thread), including a request for cross-column comparison of related financial columns.
An implementation already exists in contrib — ExpectMulticolumnValuesToBeEqual — but contrib expectations are not exported, not in the Gallery, not schema-backed, and not integration-tested, so it is effectively undiscoverable and unsupported.
Proposed capability
Promote ExpectMulticolumnValuesToBeEqual into the supported core/ namespace so it is exported, published in the Expectation Gallery, and integration-tested across all backends it runs on. Usage:
import great_expectations.expectations as gxe
gxe.ExpectMulticolumnValuesToBeEqual(column_list=["amount_a", "amount_b", "amount_c"])
Semantics: for each row, the expectation passes when all listed columns are equal (with a defined, consistent null policy — see Additional context).
Target backends (expected to run against): Pandas, Spark, and all supported SQL dialects (SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, Snowflake, Databricks, Redshift, and the PostgreSQL-family sources). The contrib metric already implements pandas, SqlAlchemy, and Spark, so this is a promotion — not new computation.
Implementation Guide
The contrib source to promote is contrib/experimental/great_expectations_experimental/expectations/expect_multicolumn_values_to_be_equal.py. The closest already-supported template to copy is great_expectations/expectations/core/expect_column_pair_values_to_be_equal.py plus its metric great_expectations/expectations/metrics/column_pair_map_metrics/column_pair_values_equal.py.
The contrib metric already has live pandas, SqlAlchemy, and Spark implementations (@multicolumn_condition_partial(engine=...) for all three). No new per-engine computation is required — the work is packaging it to the supported bar.
Definition of done:
- Class
ExpectMulticolumnValuesToBeEqual(MulticolumnMapExpectation)ingreat_expectations expectations/core/expect_multicolumn_values_to_be_equal.py, withmap_metric = "multicolumn_values.equal"(rename the contrib metric name to the dotted core convention),success_keys, andargs_keys. Do not setexpectation_type(it's derived from the class name). Drop the contrib class's leftover unusedmin_value/max_valuefields. - Metric provider moved to
great_expectations/expectations/metrics/multicolumn_map_metrics/and registered via that package's__init__.py, keeping all three engine branches. - Support metadata: module constants
EXPECTATION_SHORT_DESCRIPTION,DATA_QUALITY_ISSUES([DataQualityIssues.VALIDITY.value]),SUPPORTED_DATA_SOURCES; alibrary_metadataClassVar with"maturity": "production",has_full_test_suite: True,manually_reviewed_code: True; and aConfig.schema_extrastamping those into the schema. Copy the exact shape from the pair exemplar. - Docstring in the standard Gallery format (short description, Args, Supported Data Sources, Data Quality Issues, Example Data, passing + failing Code Examples). Build it as an f-string.
- Renderers: at least a prescriptive renderer (
_prescriptive_template+@renderer(PRESCRIPTIVE)). - Exports: add the class to both
great_expectations/expectations/core/__init__.pyandgreat_expectations/expectations/__init__.py. - Schema: add the class to the
supported_expectationslist intasks.py, then runinvoke schemas --syncfrom the repo root and commitgreat_expectations/expectations/core/schemas/ExpectMulticolumnValuesToBeEqual.json. (CI'stest_schemas_updatedfails if this is stale.) - Integration tests at
tests/integration/data_sources_and_expectations/expectations test_expect_multicolumn_values_to_be_equal.pyusing@parameterize_batch_for_data_sources(data_source_configs=ALL_DATA_SOURCES, data=DATA)for the golden path, plus passing/failing cases. PerAGENTS.md, an integration test here is required for any Expectation behavior.
Watch out for — cross-engine null semantics. The three engines currently express nulls differently (pandas nunique(dropna=False), explicit SQL and_/or_, Spark eqNullSafe). Pick one intended null policy (recommend: null == null counts as equal, null vs. non-null is unequal), make all three engines agree, and add a targeted null test across pandas/SQL/Spark to lock it down.
Testing note for community contributors. You can run pandas, sqlite, spark, postgres, and mysql locally (some via the repo's Docker Compose). BigQuery, Snowflake, Databricks, and Redshift run only in maintainer CI with credentials — write the test against the full target set, run what you can locally, and CI covers the cloud dialects. Run a single module locally by marker from the repo root, e.g. pytest tests/integration/data_sources_and_expectations/expectations/test_expect_multicolumn_values_to_be_equal.py -m unit (pandas), or -m sqlite / -m postgresql (see AGENTS.md for marker usage).