github/vscode-codeql

Test execution timeout it not properly detected

Aberta

#1.496 aberto em 5 de set. de 2022

 (2 comentários) (0 reação) (0 responsável)TypeScript (236 forks)auto 404
VSCodebuggood first issue

Métricas do repositório

Stars
 (530 estrelas)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

Describe the bug When test execution times out (after 5 minutes by default), the extension does not seem to notice that properly:

  • The "Testing" tab still shows the test as running
  • The timeout error appears (only) in the .actual file instead of the "CodeQL Tests" output (similar to #819)

Version Extension:

CodeQL extension version: 1.6.12 
CodeQL CLI version: 2.10.4 
Platform: win32 x64

VS Code:

Version: 1.71.0 (user setup)
Commit: 784b0177c56c607789f9638da7b6bf3230d47a8c
Date: 2022-09-01T07:36:10.600Z
Electron: 19.0.12
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: 10.2.154.15-electron.0
OS: Windows_NT x64 10.0.19043
Sandboxed: No

To reproduce

  1. Open the extension settings
  2. Edit "Additional Test Arguments" and add "--timeout=1"
  3. Run a test :x: The extension does not properly handle the test failure

(remember to revert your settings changes)

Expected behavior

  • The extension should properly detect that the test failed
  • The timeout error message should appear in the "CodeQL Tests" output and not in the .actual file

Additional context The .actual file contains:

Timeout (5m0s) in api-visibility-issues#09cf2679::getATypeExposingElement#6#fffffff

(with api-visibility-issues being the query name)

Running the test with codeql test run --format=json yields the following response (might helpful to adjust the parsing logic in the extension?):

{
  "test" : "<test-dir>/api-visibility-issues.qlref",
  "pass" : false,
  "failureStage" : "EXECUTION",
  "messages" : [ ],
  "evaluationMs" : 305698,
  "expected" : "<test-dir>/api-visibility-issues.expected",
  "actual" : "<test-dir>/api-visibility-issues.actual",
  "diff" : [
    "--- expected",
    "+++ actual",
    "@@ -1,30 +1,1 @@",
    "-| ...",
    "+Timeout (5m0s) in api-visibility-issues#09cf2679::getATypeExposingElement#6#fffffff"
  ]
}

(slightly minified)

Guia do colaborador