great-expectations/great_expectations

Promote ExpectMulticolumnValuesToBeEqual to a supported core Expectation

Open

#12,002 opened on Jul 24, 2026

 (3 comments) (0 reactions) (1 assignee)Python (1,436 forks)batch import
claimedgood first issuehelp wantedready-for-work🔔 reminder-sent

Repository metrics

Stars
 (9,116 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Use case

I need to assert that three or more columns hold the same value on every row — e.g. a set of amount columns that should always agree, or a value replicated across columns during a join that must stay in sync. Core ships ExpectColumnValuesToBeEqual's two-column form (ExpectColumnPairValuesToBeEqual), but there is no supported N-column generalization. This was raised in discussion #4276 (the standing "suggest an Expectation" thread), including a request for cross-column comparison of related financial columns.

An implementation already exists in contribExpectMulticolumnValuesToBeEqual — but contrib expectations are not exported, not in the Gallery, not schema-backed, and not integration-tested, so it is effectively undiscoverable and unsupported.

Proposed capability

Promote ExpectMulticolumnValuesToBeEqual into the supported core/ namespace so it is exported, published in the Expectation Gallery, and integration-tested across all backends it runs on. Usage:

import great_expectations.expectations as gxe

gxe.ExpectMulticolumnValuesToBeEqual(column_list=["amount_a", "amount_b", "amount_c"])

Semantics: for each row, the expectation passes when all listed columns are equal (with a defined, consistent null policy — see Additional context).

Target backends (expected to run against): Pandas, Spark, and all supported SQL dialects (SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, Snowflake, Databricks, Redshift, and the PostgreSQL-family sources). The contrib metric already implements pandas, SqlAlchemy, and Spark, so this is a promotion — not new computation.

Implementation Guide

The contrib source to promote is contrib/experimental/great_expectations_experimental/expectations/expect_multicolumn_values_to_be_equal.py. The closest already-supported template to copy is great_expectations/expectations/core/expect_column_pair_values_to_be_equal.py plus its metric great_expectations/expectations/metrics/column_pair_map_metrics/column_pair_values_equal.py.

The contrib metric already has live pandas, SqlAlchemy, and Spark implementations (@multicolumn_condition_partial(engine=...) for all three). No new per-engine computation is required — the work is packaging it to the supported bar.

Definition of done:

  1. Class ExpectMulticolumnValuesToBeEqual(MulticolumnMapExpectation) in great_expectations expectations/core/expect_multicolumn_values_to_be_equal.py, with map_metric = "multicolumn_values.equal" (rename the contrib metric name to the dotted core convention), success_keys, and args_keys. Do not set expectation_type (it's derived from the class name). Drop the contrib class's leftover unused min_value/max_value fields.
  2. Metric provider moved to great_expectations/expectations/metrics/multicolumn_map_metrics/ and registered via that package's __init__.py, keeping all three engine branches.
  3. Support metadata: module constants EXPECTATION_SHORT_DESCRIPTION, DATA_QUALITY_ISSUES ([DataQualityIssues.VALIDITY.value]), SUPPORTED_DATA_SOURCES; a library_metadata ClassVar with "maturity": "production", has_full_test_suite: True, manually_reviewed_code: True; and a Config.schema_extra stamping those into the schema. Copy the exact shape from the pair exemplar.
  4. Docstring in the standard Gallery format (short description, Args, Supported Data Sources, Data Quality Issues, Example Data, passing + failing Code Examples). Build it as an f-string.
  5. Renderers: at least a prescriptive renderer (_prescriptive_template + @renderer(PRESCRIPTIVE)).
  6. Exports: add the class to both great_expectations/expectations/core/__init__.py and great_expectations/expectations/__init__.py.
  7. Schema: add the class to the supported_expectations list in tasks.py, then run invoke schemas --sync from the repo root and commit great_expectations/expectations/core/schemas/ExpectMulticolumnValuesToBeEqual.json. (CI's test_schemas_updated fails if this is stale.)
  8. Integration tests at tests/integration/data_sources_and_expectations/expectations test_expect_multicolumn_values_to_be_equal.py using @parameterize_batch_for_data_sources(data_source_configs=ALL_DATA_SOURCES, data=DATA) for the golden path, plus passing/failing cases. Per AGENTS.md, an integration test here is required for any Expectation behavior.

Watch out for — cross-engine null semantics. The three engines currently express nulls differently (pandas nunique(dropna=False), explicit SQL and_/or_, Spark eqNullSafe). Pick one intended null policy (recommend: null == null counts as equal, null vs. non-null is unequal), make all three engines agree, and add a targeted null test across pandas/SQL/Spark to lock it down.

Testing note for community contributors. You can run pandas, sqlite, spark, postgres, and mysql locally (some via the repo's Docker Compose). BigQuery, Snowflake, Databricks, and Redshift run only in maintainer CI with credentials — write the test against the full target set, run what you can locally, and CI covers the cloud dialects. Run a single module locally by marker from the repo root, e.g. pytest tests/integration/data_sources_and_expectations/expectations/test_expect_multicolumn_values_to_be_equal.py -m unit (pandas), or -m sqlite / -m postgresql (see AGENTS.md for marker usage).

Contributor guide