Promote `ExpectColumnValuesToNotBeOutliers` to supported core with SQL + Spark

Use case

I want a first-class way to flag statistical outliers in a numeric column — values far from the column's center by an IQR- or standard-deviation-based threshold — as part of routine data quality and anomaly detection. This is the most-requested capability in the community anomaly-detection thread: see discussion #7161, where multiple users over ~2 years asked for outlier detection and reported the old Gallery link was dead, and it's cross-referenced from discussion #4276.

An implementation exists in contrib (expect_column_values_to_not_be_outliers) but it is Pandas-only, unexported, un-Gallery'd, untested, and therefore unavailable to the SQL and Spark users who most need it on large tables.

Proposed capability

Promote ExpectColumnValuesToNotBeOutliers into supported core/, published in the Gallery and running on Pandas, SQL, and Spark:

import great_expectations.expectations as gxe

# IQR method (default): a value is an outlier if |value - median| >= multiplier * IQR
gxe.ExpectColumnValuesToNotBeOutliers(column="amount", method="iqr", multiplier=1.5)

# Standard-deviation method: outlier if |value - mean| >= multiplier * stdev
gxe.ExpectColumnValuesToNotBeOutliers(column="amount", method="std", multiplier=3.0)

Target backends (expected to run against): Pandas, Spark, and the supported SQL dialects (SQLite, PostgreSQL, MySQL, SQL Server, BigQuery, Snowflake, Databricks, Redshift, and the PostgreSQL-family sources). The SQL and Spark implementations do not exist yet and are the core of this work.

Implementation Guide

The contrib version's SQL and Spark branches are commented-out placeholder stubs, so both must be written from scratch, and outlier logic requires a column-level aggregate (median/IQR or mean/stdev) feeding a per-row comparison across three engines.

Contrib source (Pandas + metric in one file): contrib/experimental/great_expectations_experimental/expectations/expect_column_values_to_not_be_outliers.py. Base class is ColumnMapExpectation (per-row pass/fail) but the condition depends on column-aggregate statistics — study how existing aggregate-backed map metrics compute a per-column value and broadcast it per row (compare metrics/column_aggregate_metrics/column_mean.py for the per-engine aggregate decorators: @column_aggregate_value for pandas vs. @column_aggregate_partial for SQL/Spark). Reference template for the supported packaging bar: great_expectations/expectations/core/expect_column_values_to_match_regex.py + schema.

Semantics to preserve (from the Pandas implementation):

• method="iqr" (default): outlier if |value - median| >= multiplier * IQR; multiplier=1.5.
• method="std": outlier if |value - mean| >= multiplier * stdev.
• Unknown method → raise. success_keys = ("mostly", "method", "multiplier").

Definition of done:

1. Class ExpectColumnValuesToNotBeOutliers(ColumnMapExpectation) in great_expectations/expectations/core/expect_column_values_to_not_be_outliers.py, with a dotted map_metric name and method/multiplier params as pydantic.Field(..., description=...).
2. Metric provider relocated to great_expectations/expectations/metrics/ (out of the expectation file), registered via its package __init__.py, implementing all three engines:
   • Pandas: keep the existing logic (scipy.stats.iqr, median/mean/std).
   • SQL: compute the aggregate(s) per dialect. IQR needs a median/percentile — use each dialect's percentile function (e.g. PERCENTILE_CONT), and raise NotImplementedError on any dialect that can't express it (omit that dialect from SUPPORTED_DATA_SOURCES).
   • Spark: use Spark SQL aggregate functions (percentile_approx/stddev/mean).
   • Keep pandas/SQL/Spark numerically consistent (same outlier flagged given the same data) and add a cross-engine equivalence test.
3. Dependency (no new top-level deps): the Pandas metric imports scipy at module top level. scipy is already a core requirement, so it's fine to use — but move the import inside the pandas branch rather than at module top level. Do not add any new hard/top-level dependency; if some capability ever needs one, guard it as an optional dependency via great_expectations/compatibility/ (the NotImported sentinel pattern) or propose it as a named optional install extra — never a hard requirement.
4. Null handling (correctness landmine): scipy.stats.iqr defaults to nan_policy="propagate", so a single NaN makes the IQR NaN and flags the entire column. Decide the null policy (recommend: exclude nulls from the statistic and from evaluation), implement it consistently across engines, and add an explicit null test.
5. Support metadata: module constants EXPECTATION_SHORT_DESCRIPTION, DATA_QUALITY_ISSUES ([DataQualityIssues.NUMERIC.value]), SUPPORTED_DATA_SOURCES (only the dialects you validate); a library_metadata ClassVar ("maturity": "production", has_full_test_suite: True, manually_reviewed_code: True, _library_metadata alias); and a Config.schema_extra.
6. Docstring in the standard Gallery format (f-string), and at least a prescriptive renderer (the contrib renderers are commented out).
7. Exports: add to both great_expectations/expectations/core/__init__.py and great_expectations/expectations/__init__.py.
8. Schema: add the class to the supported_expectations list in tasks.py, run invoke schemas --sync, and commit great_expectations/expectations/core/schemas/ExpectColumnValuesToNotBeOutliers.json. (CI's test_schemas_updated enforces it.)
9. Integration tests at tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_not_be_outliers.py via @parameterize_batch_for_data_sources across the full target matrix — clean data passes, injected outliers fail, both iqr and std methods, and the null case. Per AGENTS.md, an integration test here is required.

Testing note for community contributors. Pandas, sqlite, spark, postgres, and mysql run locally (some via Docker Compose); BigQuery, Snowflake, Databricks, and Redshift run only in maintainer CI with credentials. Because this issue adds SQL logic across dialects you may not be able to run locally, lean on maintainer CI for the cloud dialects — but validate the percentile/ stddev math on sqlite + postgres locally first, since dialect percentile semantics vary. Run a single module locally by marker from the repo root, e.g. pytest tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_not_be_outliers.py -m unit (pandas), or -m sqlite / -m postgresql (see AGENTS.md for marker usage).

Definition of done / self-check. There is no single "readiness" command — confirm the mechanical gates: the metric implements every backend in SUPPORTED_DATA_SOURCES; the class is in the tasks.py supported_expectations list with a committed, in-sync schema (test_schemas_updated green); integration tests pass across the target backends (locally for what you can run, CI for the cloud dialects); and invoke lint / fmt --check / type-check are clean. (Note: the older run_diagnostics() / print_diagnostic_checklist() helpers are not a valid signal here — they score off a legacy examples attribute and misreport current expectations.)