Handle dal.IsNullCondition (isNull / isNotNull) in chat joins, access policies and compare-facts

Start with the listed switches in pkg/chat/join.go, pkg/accesspolicies/fields.go, pkg/accesspolicies/run.go, and pkg/server/endpoints/compare_facts.go, then review the nearby tests and the dalgo release containing dal-go/dalgo#193. Check the optional dbcopy filter files separately. Done means null-test queries qualify joins, respect field policies and parameter checks, remain eligible for compare-facts when safe, and adapters do not silently ignore them.

Background

dal-go/dalgo#193 (with its TypeScript counterpart dal-go/dalgo-js#10) adds an explicit null test to the dalgo condition model and to DTQL:

• Go: dal.IsNullCondition, built with dal.NewIsNullCondition(expr) / dal.NewIsNotNullCondition(expr) / FieldRef.IsNull() / FieldRef.IsNotNull(); read with Operand() dal.Expression and Negated() bool; String() renders x IS NULL / x IS NOT NULL.
• DTQL: isNull: <expression> / isNotNull: <expression> anywhere a condition is allowed (where, having, inside and/or, nested queries); never in a join on list.
• Meaning: TRUE or FALSE for every input, never UNKNOWN. A field absent from a record, an explicit null and a nil pointer (dal.IsNullValue) all count as NULL. The operand is a field, a literal, arithmetic or a scalar subquery; values, star, param and a WHERE aggregate are rejected at parse.
• Why it exists: x == null is UNKNOWN in the WHERE of a joined, nested or federated-aggregate query (matches no row) but means IS NULL when a single-source query is pushed down to an adapter, so a filter that matched 49 rows alone matched 0 once joined. isNull has one meaning everywhere. (HAVING is two-valued, HAVING g == null returns the null group; that pre-existing WHERE/HAVING difference is documented in dal-go/dalgo dtql/README.md and left unchanged.)
• Rule for adapters (dal-go/dalgo docs/queries.md, "Testing for NULL"): an adapter that evaluates a query itself and cannot translate or evaluate a null test must fail the query rather than ignore the condition. Today the default arms listed below already do, so nothing is silently dropped; this issue is about supporting it.
• Not in scope: access-policy predicates (access.DocumentCondition, condeval) keep rejecting null tests on purpose (layered-ACL acceptance V02: a missing field never matches).

Line numbers below are for the default branch at the commit named in each section.

Sites (datatug/datatug-cli @ 2b6df4c0e5cec84965bf855cb5fa10ab4a1ba446, depends on dalgo v0.88.0)

1. pkg/chat/join.go:586 qualifyJoinCondition (switch :590; Comparison :591, GroupCondition :601). Prefixes unqualified FieldRefs with the root alias in the chat JOIN builder. Default errors unsupported condition %T, so a derived chat JOIN over a query with a null test fails. Add a case that qualifies Operand() with the existing qualifyJoinExpression (:622) and rebuilds with dal.NewIsNullCondition / dal.NewIsNotNullCondition (keep Negated()).
2. pkg/accesspolicies/fields.go:36-37 walkCondition, a closure inside referencedFields (:16; switch :38 with Comparison/*Comparison :41/:43 and GroupCondition/*GroupCondition :45/:47). Default errors unsupported condition %T (intentional fail-closed). referencedFields is called by hiddenReferences (:171) from pkg/accesspolicies/run.go:79, which runs only when a field allow-list policy applies, where the walker error becomes access.ErrAccessDenied: cannot verify the query against field-restricted policies. So today every query with an isNull/isNotNull is denied under a field-restricted policy. Add a case that walks Operand() with walkExpression (value and pointer forms, like Comparison) so a null test on a hidden field is reported as a hidden reference and one on an allowed field passes. Tests next to accesspolicies_test.go:268.
3. pkg/accesspolicies/run.go:154 walkParams (switch :155; Comparison :156/:161, GroupCondition :166/:170, no default). An unknown type is silently ignored, so a Param inside a null-test operand would escape the "params only on the right of a comparison" check used by misplacedParams (:121) / paramsIn (:142) and substituteParams (~:108). Visit Operand() and treat a Param there as misplaced. (DTQL no longer parses a param operand; this closes the programmatic path.)
4. pkg/server/endpoints/compare_facts.go:167 countNativeInBindings (switch :168; Comparison :169, *Comparison :179, GroupCondition :183, *GroupCondition :196; helper comparisonMentionsCohortBinding :204). An unknown type falls through to return 0, false ("not safe"), so a saved query containing a null test becomes ineligible for compare-facts. Add a case returning 0, true when the operand does not mention the cohort field/parameter and 0, false when it does.
5. Optional: pkg/dbcopy/filter/operator.go (:11-14, :38-45) lists is_null / is_not_null as deferred because dalgo lacked them; pkg/dbcopy/filter/compile.go:42 builds conditions through dal.WhereField. These can now map to dal.NewIsNullCondition / NewIsNotNullCondition.
6. Safe, no change: pkg/chat/join.go:818 and apps/datatugapp/commands/cmd_query_run_saved.go:434 only inspect join ON conditions.

Expected behavior

A saved query / chat result using isNull / isNotNull can be joined, runs under field-restricted policies exactly like the equivalent comparison would (hidden fields stay hidden), and stays eligible for compare-facts when the null test does not involve the cohort binding. Bump github.com/dal-go/dalgo to the release containing dal-go/dalgo#193 first.

Context for the chat UI: when a derived join copies the parent's filter, == null / != null should be written as isNull / isNotNull (see datatug-apps) because a joined WHERE evaluates == null as UNKNOWN.

🤖 Generated with Claude Code