Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Declarative: Multi-token authenticator with rate-limit-aware rotation

Open
  • Is something wrong?
#835 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@aaronsteers is already working on this.

Since Nov 9, 2025.

Assessment

This issue has not been assessed yet.

Description

manifest-only feature gaps

Problem

Many API connectors need to rotate through multiple authentication tokens to increase effective rate limits and avoid hitting API quotas. Currently, this requires custom Python code and cannot be implemented in declarative manifest-only connectors.

Current Workaround

Connectors like source-github implement custom authenticators in Python. See source-github/utils.py:53-186 which implements MultipleTokenAuthenticatorWithRateLimiter that:

  • Cycles through multiple tokens automatically
  • Tracks separate rate limits per token per API type (REST vs GraphQL)
  • Automatically switches tokens when limits are reached
  • Sleeps until reset if all tokens are exhausted

Proposed Solution

Add a declarative MultiTokenAuthenticator component that supports:

  1. Token rotation: Accept multiple tokens and cycle through them
  2. Rate limit tracking: Monitor rate limits per token via response headers or dedicated endpoints
  3. Separate counters: Support different rate limit buckets (e.g., REST vs GraphQL)
  4. Automatic switching: Switch to next token when current token is rate-limited
  5. Sleep/retry logic: Wait until rate limit reset when all tokens are exhausted

Example Configuration

authenticator:
  type: MultiTokenAuthenticator
  tokens:
    - "{{ config.tokens[0] }}"
    - "{{ config.tokens[1] }}"
    - "{{ config.tokens[2] }}"
  rate_limit_tracking:
    - resource: "core"  # REST API
      header_prefix: "x-ratelimit-"
    - resource: "graphql"  # GraphQL API
      header_prefix: "x-ratelimit-"
  max_wait_time: "{{ config.max_waiting_time }}"

Impact

This would enable connectors like source-github, source-gitlab, and others to be fully declarative while maintaining their multi-token rate limiting capabilities.

Related

Code References

Dominant language
Python
Stars
27
Forks
56
Avg merge
3d 18h
Merged PRs (30d)
30

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from airbytehq/airbyte-python-cdk

All issues in airbytehq/airbyte-python-cdk

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.