bug: microcks test get swallows test failure details and operation results in text output mode

Open Beginner friendly
  • Is something wrong?
#608 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
85/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
cli

Research direction

Start in cmd/testQuery.go, especially the conversion to TestResultSummary, and compare text output with cmd/testExecutor.go and the existing formatter in pkg/output. Add or update a focused test for microcks test get in text mode. Done means the output includes failed operations and their error messages, not just the summary table.

Written by the indexing model from the issue text.

Description

Describe the bug

I noticed that when running microcks test get <testResultId> on a failed test, the CLI doesn't show any failure details or which operations failed.

Right now, it just prints the exact same 1-line table you get from microcks test list:

ID      SERVICE ID  SUCCESS  IN PROGRESS  ELAPSED
test-1  svc-orders  false    false        150ms

Looking at cmd/testQuery.go, the CLI actually calls mc.GetFullTestResult(args[0]) which returns the full TestCaseResults and step error messages, but then lines 97–107 cast it down to TestResultSummary and drop all the operation results before printing:

return printTestResults([]connectors.TestResultSummary{{
    ID:             result.ID,
    Version:        result.Version,
    TestNumber:     result.TestNumber,
    TestDate:       result.TestDate,
    TestedEndpoint: result.TestedEndpoint,
    ServiceID:      result.ServiceID,
    ElapsedTime:    result.ElapsedTime,
    Success:        result.Success,
    InProgress:     result.InProgress,
}})

So if a test fails, there's no way to see what actually went wrong from the terminal without passing --output json and digging through raw JSON.

Expected behavior

In default text mode, microcks test get should format the test result using the existing output.NewFormatter(output.FormatText) (similar to what cmd/testExecutor.go does):

Test test-1: FAILURE (150ms)
  [FAIL] POST /orders
      order-validation: price: expected number got string

This way you can immediately see which operations failed and the error message directly in your terminal.

Actual behavior
$ microcks test get test-1 --microcksURL http://127.0.0.1:8888
ID      SERVICE ID  SUCCESS  IN PROGRESS  ELAPSED
test-1  svc-orders  false    false        150ms

All test cases, operations, and failure messages are omitted from the output.

Image
How to Reproduce?
  1. Spin up a quick local mock returning a failed test with testCaseResults:
python3 -c '
from http.server import HTTPServer, BaseHTTPRequestHandler
import json
class Mock(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.end_headers()
        self.wfile.write(json.dumps({
            "id": "test-1",
            "serviceId": "svc-orders",
            "success": False,
            "elapsedTime": 150,
            "testCaseResults": [{
                "operationName": "POST /orders",
                "success": False,
                "elapsedTime": 150,
                "testStepResults": [{
                    "requestName": "order-validation",
                    "success": False,
                    "message": "price: expected number got string"
                }]
            }]
        }).encode())
    def log_message(self, *args): pass
HTTPServer(("127.0.0.1", 8888), Mock).serve_forever()
'
  1. Run test get:
go run main.go test get test-1 --microcksURL http://127.0.0.1:8888

You'll see it only prints the 1-line summary table without showing the failed POST /orders operation or the error message.

Microcks version or git rev

main

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

cli

Additional information

Since output.NewFormatter(output.FormatText) is already implemented in pkg/output, we can just route text mode through it in newTestGetCommand.

Tested this locally and happy to submit a PR with the fix and unit test if this sounds good!

Dominant language
Go
Stars
59
Forks
74
Avg merge
4d 5h
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microcks/microcks-cli

All issues in microcks/microcks-cli

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.