Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

In workflow, enrichment listed in apply are not replayed if they are listed in skipped

Open
  • Is something wrong?
#438 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
go
Domain
stream-processing

Research direction

Start by locating skipFromMeta and read the workflow tests named TestWorkflows, TestWorkflowsDeps, TestWorkflowsParallel, and TestWorkflowsWithOrderResources. Reproduce the case where apply contains previously skipped enrichments, then verify that those enrichments are replayed and appear in the resulting document, alongside the existing failed-enrichment behavior.

Written by the indexing model from the issue text.

Description

Hey, I made this PR to fix the issue that I currently have, it's related with this Slack Thread.

Currently have an issue, all enrichments listed in apply meta are not replayed. We added this processor to replay all enrichment, even skipped ones because, some of them can be skipped because of a dependencies issue with the failed one. We added this processor before the workflow:

- label: replay_skipped
  switch:
    - check: this.meta.enrichments.exists("skipped") && this.meta.enrichments.skipped.length() > 0
      processors:
        - label: replay_skipped_enrichments
          bloblang: |-
            root = this
            root.meta.enrichments.apply = this.meta.enrichments.failed.keys().merge(this.meta.enrichments.skipped)

Our workflow looks like that

---
processor_resources:
  - label: enrichments
    workflow:
      meta_path: meta.enrichments
      order:
...

But, in the document resulted, the enrichments listed in apply are not replayed, only the one previously in failed is replayed. They are not listed in succeeded, they are in skipped and the field added by enrichment are not present.

I'm not expert in Go but in my PR, if I well understand, in function skipFromMeta, the if skipped section should be before the if apply one because the if apply will remove the enrichment from the list skipList filled by the if skipped section.

I wrote test to validate if we have the expected result and it's works 🍾

I run tests for others functions TestWorkflows, TestWorkflowsDeps, TestWorkflowsParallel and TestWorkflowsWithOrderResources

Dominant language
Go
Stars
571
Forks
121
Avg merge
1d 13h
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from redpanda-data/benthos

All issues in redpanda-data/benthos

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.