Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[request] add schedule processor

Open
  • Is something wrong?
#273 10 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
go
Domain
stream-processing

Research direction

Start by reviewing the existing generate input and read_until behavior described in the example, then compare the two proposed scheduling approaches: nesting an input under generate or adding a schedule input. Done means a tested way to repeat child input on an interval or cron without an external scheduler, including the paginated HTTP use case.

Written by the indexing model from the issue text.

Description

Hi,
It would be nice to add schedule processor to consume the child input with defined interval or cron.

Use case: read paginated http response with interval.

I can do it only with external scheduler (like cron) now. Or maybe is it possible with current capabilities?

Example:

input:
  # need repeat this block every hour
  read_until:
    check: '@offset >= 5'
    input:
      generate:
        count: 0
        interval: 1s
        mapping: 'root = null'
      processors:
        - mapping: |
            meta n = 5
            meta offset = counter(min: 0)
        - http:
            url: https://httpbin.org/links/${! metadata("n") }/${! metadata("offset") }
            verb: GET

pipeline:
  processors:
    - xml:
        operator: to_json
    - mapping: |
        root = json("html.body.a")
    - unarchive:
        format: json_array
    - mutation: |
        root.page = @offset

output:
  stdout: {}

Possible solutions:

  • add input field to the generate input
  • add schedule input
Dominant language
Go
Stars
571
Forks
121
Avg merge
1d 13h
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from redpanda-data/benthos

All issues in redpanda-data/benthos

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.