github/vscode-codeql

Test execution timeout it not properly detected

Ouverte

#1 496 ouverte le 5 sept. 2022

 (2 commentaires) (0 réaction) (0 personne assignée)TypeScript (236 forks)auto 404
VSCodebuggood first issue

Métriques du dépôt

Stars
 (530 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

Describe the bug When test execution times out (after 5 minutes by default), the extension does not seem to notice that properly:

  • The "Testing" tab still shows the test as running
  • The timeout error appears (only) in the .actual file instead of the "CodeQL Tests" output (similar to #819)

Version Extension:

CodeQL extension version: 1.6.12 
CodeQL CLI version: 2.10.4 
Platform: win32 x64

VS Code:

Version: 1.71.0 (user setup)
Commit: 784b0177c56c607789f9638da7b6bf3230d47a8c
Date: 2022-09-01T07:36:10.600Z
Electron: 19.0.12
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: 10.2.154.15-electron.0
OS: Windows_NT x64 10.0.19043
Sandboxed: No

To reproduce

  1. Open the extension settings
  2. Edit "Additional Test Arguments" and add "--timeout=1"
  3. Run a test :x: The extension does not properly handle the test failure

(remember to revert your settings changes)

Expected behavior

  • The extension should properly detect that the test failed
  • The timeout error message should appear in the "CodeQL Tests" output and not in the .actual file

Additional context The .actual file contains:

Timeout (5m0s) in api-visibility-issues#09cf2679::getATypeExposingElement#6#fffffff

(with api-visibility-issues being the query name)

Running the test with codeql test run --format=json yields the following response (might helpful to adjust the parsing logic in the extension?):

{
  "test" : "<test-dir>/api-visibility-issues.qlref",
  "pass" : false,
  "failureStage" : "EXECUTION",
  "messages" : [ ],
  "evaluationMs" : 305698,
  "expected" : "<test-dir>/api-visibility-issues.expected",
  "actual" : "<test-dir>/api-visibility-issues.actual",
  "diff" : [
    "--- expected",
    "+++ actual",
    "@@ -1,30 +1,1 @@",
    "-| ...",
    "+Timeout (5m0s) in api-visibility-issues#09cf2679::getATypeExposingElement#6#fffffff"
  ]
}

(slightly minified)

Guide contributeur