Standard representation for assertion expected and actual values

Ouverte
  • Quelque chose ne va pas ?
#67 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@Ma11hewThomas y travaille déjà.

Depuis le 31/10/2026.

  • #87 par @Ma11hewThomas — ouverte

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
30/100
Type d'issue
Fonctionnalité
Clarté
À clarifier
Activité
Active
Stack technique
json
Domaine
testing-qa

Piste de recherche

Start with the linked JsonTestWriter.cs locations for normal and retry attempts, then review CTRF’s current specification and schema alongside issue #53. Compare how expected and actual values are represented by Microsoft.Testing.Platform and identify the schema decisions the issue asks for. Done means a maintainer-approved direction for structured assertion data or documented guidance for using extra.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Producer context

Microsoft.Testing.Platform has a first-class assertion-failure payload with separate Expected and Actual strings (AssertionFailureProperty). Its terminal and protocol reporters preserve those values so consumers can render a useful diff.

The CTRF reporter currently cannot preserve that structure. It emits only the failure message and trace defined by CTRF:

We searched the current specification, schema, issues, discussions, and pull requests and found no standardized expected/actual assertion fields. Issue #53 also indicates a preference to promote broadly useful concepts to first-class CTRF fields rather than create incompatible extra conventions.

Questions

  1. Does CTRF want first-class structured assertion data, such as optional expected and actual fields on both the Test object and Attempt History Entry object?
  2. If so, should values be strings (matching many framework protocols), arbitrary JSON values, or a richer assertion/diff object?
  3. How should multiple assertion failures from one test be represented: only the primary failure, or an ordered array of assertion failures?
  4. If this is intentionally outside the core schema, is the recommended mapping a producer-namespaced object under extra, and should CTRF document a suggested semantic shape so producers remain interoperable?

Why this matters

Flattening expected/actual into message loses machine-readable comparison data and forces every CTRF consumer to parse framework-specific prose. Keeping it only in a producer-specific extra shape preserves the data but fragments interoperability. Guidance on the intended level would let MTP map the existing structured assertion payload correctly without prematurely prescribing a schema design.

Langage dominant
Aucune donnée de langage
Étoiles
103
Forks
4
Merge moyen
1 h 3 min
PR mergées (30 j)
2

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de ctrf-io/ctrf

Toutes les issues de ctrf-io/ctrf

Issues similaires

Plus d'issues Testing & QA

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.