email_outputs should gracefully handle missing workflow_id

Open Beginner friendly
#533 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
64/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
tooling

Research direction

Start in email_outputs.py at process_workflow around line 367, where the traceback shows the missing workflowId is accessed. Trace how process_submission passes submission data into this entry point, then verify that a submission without workflowId is skipped and processing can continue or retry as intended. Confirm the behavior with the repository's existing tests, if applicable.

Written by the indexing model from the issue text.

Description

Seen returned from Terra, a submission that didn't have a workflowId:

Traceback (most recent call last):
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 438, in run_with_retries
func()
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 141, in <lambda>
lambda: send_workspace_emails(system_clients, gcloud_clients_list, email_templates_list, submission_filters),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 306, in send_workspace_emails
process_workspace(system_clients, gcloud_clients, email_templates_list, workspace, submission_filters)
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 324, in process_workspace
process_submission(system_clients, gcloud_clients, email_templates_list, submission_info)
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 351, in process_submission
process_workflow(
File "/elided/pythonuserbase/lib/python3.12/site-packages/dropseq_terra_utils/email_outputs.py", line 367, in process_workflow
workflow_id = submission_workflow["workflowId"]
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'workflowId'

Not sure what other keys might be missing. But at least for this missing workflowId case it's probably ok to just skip this submission and try again later. If we find ourselves infinitely retrying due to perpetually malformed responses we can file another ticket and/or contact Terra support.

Dominant language
Jupyter Notebook
Stars
136
Forks
34
Avg merge
6m
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from broadinstitute/Drop-seq

All issues in broadinstitute/Drop-seq

Similar issues

More DevTools issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.