Instance matching treats unknown types as apart from every instance

Start with types_match, match_instance, and the solver paths match_provided and match_compiler_instance in compiler-frontend/checking/src/core/constraint/matching.rs and constraint.rs. Read the existing matching and chain behavior, then add the two described compiler fixtures and capture their current output before changing matching. Run just t compiler; done means unknown types no longer hide known mismatches or trigger errors through a skipped chain member, with only expected diagnostics changed.

Problem

Instance matching treats an unknown type as apart from every instance pattern, so the solver can't tell whether a failed match depends on the unknown type. Unknown types replace types whose errors were already reported, e.g., the invalid application in type Bad = Int String. As a result, the solver hides some instance errors that don't depend on the unknown type, and it reports some follow-up errors that do.

Independent instance errors are hidden

Since #646, the solver drops every unsolved constraint that mentions an unknown type and marks its evidence as an error without a report. A review comment on #646 pointed out that the rule also drops constraints that fail because of a known argument. For example, with only instance C Int Int, the constraint C Bad Boolean fails because Boolean can't match Int, whatever Bad means.

module Main where

class C :: Type -> Type -> Constraint
class C a b

instance C Int Int

type Bad = Int String

needsKnownMismatch :: C Bad Boolean => Int
needsKnownMismatch = 0

usesKnownMismatch :: Int
usesKnownMismatch = needsKnownMismatch

needsUnknownOnly :: C Bad Int => Int
needsUnknownOnly = 0

usesUnknownOnly :: Int
usesUnknownOnly = needsUnknownOnly

needsKnown :: C Int Boolean => Int
needsKnown = 0

usesKnown :: Int
usesKnown = needsKnown

viaArgument :: forall a. C a Boolean => a -> Int
viaArgument _ = 0

usesViaArgument :: Bad -> Int
usesViaArgument value = viaArgument value

I ran the fixture on main at 3284cb224 and on b487218a4, which is main just before #646 merged.

The C Bad Boolean error doesn't depend on the unknown type, so main should still report it. main still reports the InvalidTypeApplication error on Bad, so the module fails to compile. The hidden error appears once Bad is fixed.

Instance chains skip members that might match

An unknown argument can also make an instance chain skip the member that the user meant, so the chain commits to a later member. The later member can then report an error that the unknown type caused.

module Main where

import Prim.TypeError (class Fail, Text)

class D :: Type -> Type -> Constraint
class D a b

instance D Int b
else instance Fail (Text "fell through to the catch-all") => D a b

type Bad = Int String

needsChain :: D Bad Boolean => Int
needsChain = 0

usesChain :: Int
usesChain = needsChain

main reports the following error after the InvalidTypeApplication error on Bad.

Error! · [CustomFailure] · Main.purs:18:13
  •
  │
  • fell through to the catch-all
  │
  │   usesChain = needsChain
  │               ╰─────────
  •

With D Int Boolean in place of D Bad Boolean, the first member matches and the module reports no errors. b487218a4 reports the same CustomFailure, so the behavior predates #646. The guard from #646 can't catch the error, because the catch-all member solves the constraint and the guard only sees unsolved constraints.

Cause

types_match has no case for Type::Unknown, so an unknown type on either side falls through to (_, _) => Ok(MatchType::Apart). match_instance then returns Apart at the first apart argument that functional dependencies don't determine. For C Bad Boolean, it stops at Bad and never compares Boolean with Int.

The solver treats Apart as a reason to try the next chain member, which explains the chain behavior. After matching, the guard from #646 can only check whether the constraint contains an unknown type, because matching doesn't record why it failed.

Proposed direction

Fix the cause in matching instead of refining the guard. types_match could return a new result for unknown types, similar to MatchType::Stuck, meaning that the types might match but the solver can't commit to the instance. The solver would then do the following.

• Stop at a chain member that only an unknown type blocks.
• Report an unsolved constraint when every candidate is apart on known arguments.
• Drop an unsolved constraint without a report when an unknown type blocks one of its candidates.

The contains_unknown guard could then use the result that matching recorded, or be removed. The change also needs to cover the following cases.

• Given constraints in match_provided.
• Argument positions that functional dependencies determine.
• Classes that the compiler solves in match_compiler_instance.

Also check how a reported constraint prints. Before #646, the instantiation case printed C (Int String) Boolean. Bad now expands to an unknown type, so the new message may show an unknown type placeholder instead.

Tests

Add both reproductions as compiler fixtures, and commit them with the current output first so that the fix commit shows the change in diagnostics. Then run just t compiler and confirm that only the expected diagnostics change.

Priority

The priority is low, because both effects only happen in modules that already report an error. Neither effect lets an invalid module compile.

Investigation: https://ampcode.com/threads/T-01a13033-7f9a-767c-aaae-c2eb0a8fa390