`POST /jobs` internal server error if validator references unspecified optional parameter

Open
  • Is something wrong?
#2,516 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
aws, flask, python
Domain
api, backend, databases

Research direction

Start with apps/api/src/hyp3_api/handlers.py, apps/api/src/hyp3_api/validation.py, and lib/dynamo/dynamo/jobs.py; trace how POST /jobs prepares parameters and runs validation. Reproduce with the supplied payload that omits bounds, then check validation runs against prepared jobs and the request no longer fails with a KeyError.

Written by the indexing model from the issue text.

Description

Jira Bug

Jira: https://asfdaac.atlassian.net/browse/TOOL-3335

Note: The above link is accessible only to members of ASF.

The POST /jobs endpoint returns an internal server error if a validator attempts to reference an optional job parameter that was not provided. For example, this can be reproduced by attempting to submit either SRG_GSLC or SRG_TIME_SERIES without the optional bounds parameter. Here's an example payload that should validate successfully:

{
  "validate_only": true,
  "jobs": [
    {
      "job_type": "SRG_GSLC",
      "name": "Job Name",
      "job_parameters": {
        "granules": [
          "S1A_IW_RAW__0SDV_20231229T134339_20231229T134411_051870_064437_4F42",
          "S1A_IW_RAW__0SDV_20231229T134404_20231229T134436_051870_064437_5F38"
        ],
        "bounds": [
          -116.583,
          35.714,
          -113.209,
          38.138
        ]
      }
    }
  ]
}

and here's the same payload without bounds, which should return a 500 error:

{
  "validate_only": true,
  "jobs": [
    {
      "job_type": "SRG_GSLC",
      "name": "Job Name",
      "job_parameters": {
        "granules": [
          "S1A_IW_RAW__0SDV_20231229T134339_20231229T134411_051870_064437_4F42",
          "S1A_IW_RAW__0SDV_20231229T134404_20231229T134436_051870_064437_5F38"
        ]
      }
    }
  ]
}

The API Lambda logs show:

[ERROR]	2024-12-05T00:35:11.108Z	ea26392c-cff8-4ddc-8021-32c36437b530	Exception on /jobs [POST]
Traceback (most recent call last):
  File "/var/task/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/task/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/task/flask_cors/extension.py", line 194, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/var/task/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/task/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/var/task/openapi_core/contrib/flask/decorators.py", line 52, in decorated
    response = self.handle_request(
  File "/var/task/openapi_core/unmarshalling/processors.py", line 28, in handle_request
    return valid_handler(request_unmarshal_result)
  File "/var/task/openapi_core/contrib/flask/handlers.py", line 75, in __call__
    rv = self.view(*self.view_args, **self.view_kwargs)
  File "/var/task/hyp3_api/routes.py", line 136, in jobs_post
    return jsonify(handlers.post_jobs(request.get_json(), g.user))
  File "/var/task/hyp3_api/handlers.py", line 28, in post_jobs
    validate_jobs(body['jobs'])
  File "/var/task/hyp3_api/validation.py", line 221, in validate_jobs
    validator(job, job_granule_metadata)
  File "/var/task/hyp3_api/validation.py", line 144, in check_bounds_formatting
    bounds = job['job_parameters']['bounds']
KeyError: 'bounds'

The problem is that the post_jobs handler in handlers.py calls validate_jobs before calling dynamo.jobs.put_jobs, which is responsible for populating default parameter values (see jobs.py). The fix might be as simple as calling validate_jobs on the prepared jobs in dynamo.jobs.put_jobs before writing them to the database.

This could potentially affect any job type, but after reviewing validation.py, I'm not seeing any other job types that would currently be affected, so this should only affect the LAVAS deployments right now.

Dominant language
Python
Stars
53
Forks
13
Avg merge
1d 2h
Merged PRs (30d)
20

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ASFHyP3/hyp3

All issues in ASFHyP3/hyp3

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.