microsoft/restler-fuzzer

Improve logging/error-reporting when Duration is set and reached in Test or FuzzLean runs

Open

#216 opened on May 20, 2021

View on GitHub
 (1 comment) (1 reaction) (1 assignee)Python (329 forks)auto 404
help wantedux

Repository metrics

Stars
 (2,929 stars)
PR merge metrics
 (PR metrics pending)

Description

Right now RESTler throws an error is specified duration is exceeded during Test or TestFuzzLean runs.

It is "known" that Test and TestFuzzLean are meant for running till exhaustion rather than till duration timer is expired. But since RESTler allows duration to be specified when Test or TestFuzzLean modes are used - it would be better for users to have a clear log message explaining that, rather than a RESTLer engine failure error.

Contributor guide

Research direction
Look at the engine code where duration is checked. Add a conditional that if the run mode is Test or FuzzLean, log a warning instead of raising an error when the duration expires. Modify the relevant error handler to output a clear message explaining that these modes are intended for exhaustive runs and the duration limit is ignored.
Tech stack
python
Domain
backendtesting qaapi
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
1
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
85