Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Version dependence in the test suite

Aberta
#2,069 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
35/100
Tipo de issue
Bug
Clareza
Precisa de esclarecimento
Status de atividade
Estagnada
Stack de tecnologia
python
Domínio
testing-qa

Direção de pesquisa

Execute a suíte de testes de conformidade com Python 3.14 e compare os resultados em conformance/results/mypy/ e conformance/results/pyright/. Leia os casos referenciados, incluindo classes_classvar.py, dataclasses_final.py, generics_variance_inference.py, qualifiers_final_annotation.py e annotations_forward_refs.py, e então determine como os resultados esperados devem levar em conta as diferenças entre as versões compatíveis. O trabalho estará concluído quando os resultados da suíte dependentes da versão estiverem representados de forma intencional e consistente.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

topic: other

While working on #2068 I ran the conformance test suite under Python 3.14 and got a number of changes:

diff --git a/conformance/results/mypy/classes_classvar.toml b/conformance/results/mypy/classes_classvar.toml
index 0072d63..ce30b65 100644
--- a/conformance/results/mypy/classes_classvar.toml
+++ b/conformance/results/mypy/classes_classvar.toml
@@ -12,7 +12,6 @@ classes_classvar.py:40: error: Name "var" is not defined  [name-defined]
 classes_classvar.py:45: error: ClassVar cannot contain type variables  [misc]
 classes_classvar.py:46: error: ClassVar cannot contain type variables  [misc]
 classes_classvar.py:52: error: Incompatible types in assignment (expression has type "dict[Never, Never]", variable has type "list[str]")  [assignment]
-classes_classvar.py:54: error: Variable should not be annotated with both ClassVar and Final  [misc]
 classes_classvar.py:55: error: Invalid type: ClassVar nested inside other type  [valid-type]
 classes_classvar.py:67: error: Invalid type: ClassVar nested inside other type  [valid-type]
 classes_classvar.py:69: error: ClassVar can only be used for assignments in class body  [misc]
@@ -31,5 +30,6 @@ classes_classvar.py:140: note: Protocol member ProtoA.y expected class variable,
 conformance_automated = "Fail"
 errors_diff = """
 Line 47: Expected 1 errors
+Line 54: Expected 1 errors
 Line 67: Unexpected errors ['classes_classvar.py:67: error: Invalid type: ClassVar nested inside other type  [valid-type]']
 """
diff --git a/conformance/results/mypy/dataclasses_final.toml b/conformance/results/mypy/dataclasses_final.toml
index fdf8691..eb91705 100644
--- a/conformance/results/mypy/dataclasses_final.toml
+++ b/conformance/results/mypy/dataclasses_final.toml
@@ -7,12 +7,10 @@ conformance_automated = "Fail"
 errors_diff = """
 Line 27: Expected 1 errors
 Line 16: Unexpected errors ['dataclasses_final.py:16: error: Final name must be initialized with a value  [misc]']
-Line 18: Unexpected errors ['dataclasses_final.py:18: error: Final can be only used as an outermost qualifier in a variable annotation  [valid-type]']
 Line 24: Unexpected errors ['dataclasses_final.py:24: error: Expression is of type "Any", not "int"  [assert-type]']
 """
 output = """
 dataclasses_final.py:16: error: Final name must be initialized with a value  [misc]
-dataclasses_final.py:18: error: Final can be only used as an outermost qualifier in a variable annotation  [valid-type]
 dataclasses_final.py:24: error: Expression is of type "Any", not "int"  [assert-type]
 dataclasses_final.py:35: error: Cannot assign to final attribute "final_no_default"  [misc]
 dataclasses_final.py:36: error: Cannot assign to final attribute "final_with_default"  [misc]
diff --git a/conformance/results/mypy/generics_variance_inference.toml b/conformance/results/mypy/generics_variance_inference.toml
index b1158c7..ef877e1 100644
--- a/conformance/results/mypy/generics_variance_inference.toml
+++ b/conformance/results/mypy/generics_variance_inference.toml
@@ -6,6 +6,7 @@ generics_variance_inference.py:28: error: Incompatible types in assignment (expr
 generics_variance_inference.py:41: error: Incompatible types in assignment (expression has type "ShouldBeCovariant1[float]", variable has type "ShouldBeCovariant1[int]")  [assignment]
 generics_variance_inference.py:49: error: Incompatible types in assignment (expression has type "ShouldBeCovariant2[float]", variable has type "ShouldBeCovariant2[int]")  [assignment]
 generics_variance_inference.py:58: error: Incompatible types in assignment (expression has type "ShouldBeCovariant3[float]", variable has type "ShouldBeCovariant3[int]")  [assignment]
+generics_variance_inference.py:66: error: Incompatible types in assignment (expression has type "ShouldBeCovariant4[int]", variable has type "ShouldBeCovariant4[float]")  [assignment]
 generics_variance_inference.py:67: error: Incompatible types in assignment (expression has type "ShouldBeCovariant4[float]", variable has type "ShouldBeCovariant4[int]")  [assignment]
 generics_variance_inference.py:80: error: Incompatible types in assignment (expression has type "ShouldBeCovariant5[float]", variable has type "ShouldBeCovariant5[int]")  [assignment]
 generics_variance_inference.py:96: error: Incompatible types in assignment (expression has type "ShouldBeInvariant1[int]", variable has type "ShouldBeInvariant1[float]")  [assignment]
@@ -24,6 +25,7 @@ generics_variance_inference.py:170: error: Incompatible types in assignment (exp
 generics_variance_inference.py:181: error: Incompatible types in assignment (expression has type "ShouldBeCovariant6[float]", variable has type "ShouldBeCovariant6[int]")  [assignment]
 generics_variance_inference.py:194: error: Incompatible types in assignment (expression has type "ShouldBeContravariant2[int]", variable has type "ShouldBeContravariant2[float]")  [assignment]
 """
-conformance_automated = "Pass"
+conformance_automated = "Fail"
 errors_diff = """
+Line 66: Unexpected errors ['generics_variance_inference.py:66: error: Incompatible types in assignment (expression has type "ShouldBeCovariant4[int]", variable has type "ShouldBeCovariant4[float]")  [assignment]']
 """
diff --git a/conformance/results/mypy/qualifiers_final_annotation.toml b/conformance/results/mypy/qualifiers_final_annotation.toml
index 2d0cea3..8c0830f 100644
--- a/conformance/results/mypy/qualifiers_final_annotation.toml
+++ b/conformance/results/mypy/qualifiers_final_annotation.toml
@@ -20,8 +20,6 @@ qualifiers_final_annotation.py:71: error: Cannot assign to final name "RATE"  [m
 qualifiers_final_annotation.py:81: error: Cannot assign to final attribute "DEFAULT_ID"  [misc]
 qualifiers_final_annotation.py:94: error: Cannot assign to final name "BORDER_WIDTH"  [misc]
 qualifiers_final_annotation.py:96: error: Cannot assign to final name "__private"  [misc]
-qualifiers_final_annotation.py:107: error: Final can be only used as an outermost qualifier in a variable annotation  [valid-type]
-qualifiers_final_annotation.py:108: error: Variable should not be annotated with both ClassVar and Final  [misc]
 qualifiers_final_annotation.py:118: error: Final can be only used as an outermost qualifier in a variable annotation  [valid-type]
 qualifiers_final_annotation.py:121: error: Final can be only used as an outermost qualifier in a variable annotation  [valid-type]
 qualifiers_final_annotation.py:131: error: Invalid "NamedTuple()" field name  [misc]
@@ -40,6 +38,8 @@ qualifiers_final_annotation.py:170: error: Cannot assign to final name "PI"  [mi
 """
 conformance_automated = "Fail"
 errors_diff = """
+Line 107: Expected 1 errors
+Line 108: Expected 1 errors
 Line 149: Expected 1 errors
 Line 59: Unexpected errors ['qualifiers_final_annotation.py:59: error: Cannot assign to final attribute "ID6"  [misc]']
 Line 96: Unexpected errors ['qualifiers_final_annotation.py:96: error: Cannot assign to final name "__private"  [misc]']
diff --git a/conformance/results/pyright/annotations_forward_refs.toml b/conformance/results/pyright/annotations_forward_refs.toml
index ca35b5b..b91c665 100644
--- a/conformance/results/pyright/annotations_forward_refs.toml
+++ b/conformance/results/pyright/annotations_forward_refs.toml
@@ -1,7 +1,5 @@
 conformant = "Pass"
 output = """
-annotations_forward_refs.py:22:7 - error: "ClassA" is not defined (reportUndefinedVariable)
-annotations_forward_refs.py:23:12 - error: "ClassA" is not defined (reportUndefinedVariable)
 annotations_forward_refs.py:24:7 - error: Union syntax cannot be used with string operand; use quotes around entire expression (reportGeneralTypeIssues)
 annotations_forward_refs.py:25:13 - error: Union syntax cannot be used with string operand; use quotes around entire expression (reportGeneralTypeIssues)
 annotations_forward_refs.py:41:10 - error: Call expression not allowed in type expression (reportInvalidTypeForm)
@@ -29,11 +27,13 @@ annotations_forward_refs.py:52:11 - error: Unary operator not allowed in type ex
 annotations_forward_refs.py:53:11 - error: Binary operator not allowed in type expression (reportInvalidTypeForm)
 annotations_forward_refs.py:54:11 - error: Type expressions cannot use format string literals (f-strings) (reportGeneralTypeIssues)
 annotations_forward_refs.py:55:10 - error: Module cannot be used as a type (reportGeneralTypeIssues)
-annotations_forward_refs.py:66:26 - error: "ClassB" is not defined (reportUndefinedVariable)
 annotations_forward_refs.py:80:14 - error: Type of "ClassF" could not be determined because it refers to itself (reportGeneralTypeIssues)
 annotations_forward_refs.py:80:14 - error: Variable not allowed in type expression (reportInvalidTypeForm)
-annotations_forward_refs.py:89:8 - error: Expected class but received "(self: Self@ClassD) -> None" (reportGeneralTypeIssues)
 """
-conformance_automated = "Pass"
+conformance_automated = "Fail"
 errors_diff = """
+Line 22: Expected 1 errors
+Line 23: Expected 1 errors
+Line 66: Expected 1 errors
+Line 89: Expected 1 errors
 """
diff --git a/conformance/results/pyright/generics_variance_inference.toml b/conformance/results/pyright/generics_variance_inference.toml
index 9a378ad..5c0f32e 100644
--- a/conformance/results/pyright/generics_variance_inference.toml
+++ b/conformance/results/pyright/generics_variance_inference.toml
@@ -23,10 +23,12 @@ generics_variance_inference.py:58:35 - error: Type "ShouldBeCovariant3[float]" i
   "ShouldBeCovariant3[float]" is not assignable to "ShouldBeCovariant3[int]"
     Type parameter "T@ShouldBeCovariant3" is covariant, but "float" is not a subtype of "int"
       "float" is not assignable to "int" (reportAssignmentType)
+generics_variance_inference.py:66:36 - error: Type "ShouldBeCovariant4[int]" is not assignable to declared type "ShouldBeCovariant4[float]"
+  "ShouldBeCovariant4[int]" is not assignable to "ShouldBeCovariant4[float]"
+    Type parameter "T@ShouldBeCovariant4" is invariant, but "int" is not the same as "float" (reportAssignmentType)
 generics_variance_inference.py:67:34 - error: Type "ShouldBeCovariant4[float]" is not assignable to declared type "ShouldBeCovariant4[int]"
   "ShouldBeCovariant4[float]" is not assignable to "ShouldBeCovariant4[int]"
-    Type parameter "T@ShouldBeCovariant4" is covariant, but "float" is not a subtype of "int"
-      "float" is not assignable to "int" (reportAssignmentType)
+    Type parameter "T@ShouldBeCovariant4" is invariant, but "float" is not the same as "int" (reportAssignmentType)
 generics_variance_inference.py:80:34 - error: Type "ShouldBeCovariant5[float]" is not assignable to declared type "ShouldBeCovariant5[int]"
   "ShouldBeCovariant5[float]" is not assignable to "ShouldBeCovariant5[int]"
     Type parameter "T@ShouldBeCovariant5" is covariant, but "float" is not a subtype of "int"
@@ -80,6 +82,7 @@ generics_variance_inference.py:194:37 - error: Type "ShouldBeContravariant2[int]
     Type parameter "T@ShouldBeContravariant2" is contravariant, but "int" is not a supertype of "float"
       "float" is not assignable to "int" (reportAssignmentType)
 """
-conformance_automated = "Pass"
+conformance_automated = "Fail"
 errors_diff = """
+Line 66: Unexpected errors ['generics_variance_inference.py:66:36 - error: Type "ShouldBeCovariant4[int]" is not assignable to declared type "ShouldBeCovariant4[float]"']
 """

Analysis:

  • In classes_classvar.py, mypy allows Final[ClassVar[int]] only in 3.13+. This is in line with the runtime behavior, as this construct throws a runtime error in 3.12 and earlier. Interestingly, mypy also disallows this when from __future__ import annotations is used (so there wouldn't be a runtime error). Mypy seems to be in the wrong here according to the spec: we still explicitly disallow Final + ClassVar except in dataclasses, even if it works at runtime.
  • qualifiers_final_annotation.py also has mypy allowing a Final/ClassVar it shouldn't.
  • dataclasses_final.py is a similar issue with mypy, but here it's a positive; it correctly allows Final + ClassVar in a dataclass.
  • In generics_variance_inference.py, both mypy and pyright now infer a generic frozen dataclass as invariant. This is due to https://discuss.python.org/t/make-replace-stop-interfering-with-variance-inference/96092 .
  • In annotations_forward_refs.py, pyright changes its behavior regarding names in annotations. This is mostly correct and in line with PEP 649, but one of the new errors (the last line) seems wrong to me.

Regardless of the specifics of these changes, the general lesson is that the behavior of the test suite depends on the Python version, and we need to make sure we use a consistent Python version to run the test suite (presumably, the most recent stable version, so currently 3.13). Optionally, we could also extend the framework to support checking some tests under a more specific Python version, so we can test version-dependent behavior.

Linguagem predominante
Python
Estrelas
1.8k
Forks
302
Merge médio
23h
PRs com merge (30d)
8

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de python/typing

Todas as issues de python/typing

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.