Reduce impact of ExplainCostsRouter and many concurrent queries
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- rust
- Domain
- backend, distributed-systems
Research direction
Start by tracing ExplainCostsRouter and the existing trino-lb explain-query routing and cluster configuration entry points. Determine how users can select a cluster group and how explain queries are queued without creating routing loops. Done means explain traffic no longer sends every request to one underlying cluster and concurrent queries are protected from causing a denial of service.
Written by the indexing model from the issue text.
Description
Currently 100,000 queries in trino-lb running through ExplainCostsRouter cause 100,000 queries on the underlying Trino cluster used for the query estimations, which can lead to DOS.
To prevent this we need to let users specify the cluster group used for the explain queries instead of a single cluster. This should also take care of queuing explain queries.
An alternative solution could be to enter trino-lb itself as the explain target cluster, but users would need to be careful to not creating routing loops 🤷♂️ To prevent this a dedicated trino-lb only for explain queries could be set up. Seems unnecessary complicated though.
- Dominant language
- Rust
- Stars
- 38
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stackabletech/trino-lb
-
trino-lb is stuck after redis master shuts downPossibly taken @soenkeliebau claimed this 115 days ago. Opencustomer-request
Difficulty 4/5 3-5 days Newbie friendliness 42/100
stackabletech/trino-lb#109 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
stackabletech/trino-lb#105 ·
-
customer-request
Difficulty 5/5 Over a week Newbie friendliness 30/100
stackabletech/trino-lb#93 · 4 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 38/100
stackabletech/trino-lb#83 · 11 comments ·
-
Dependency DashboardOpen
Difficulty 4/5 3-5 days Newbie friendliness 20/100
stackabletech/trino-lb#31 ·
All issues in stackabletech/trino-lb
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
[Bug]: Bedrock request metadata forwarding does not work for /embeddingsPossibly taken A pull request linked to this issue is open or already merged. Openbug llm translation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
pytest plugin: a crashed xdist worker aborts the whole session with INTERNALERRORPossibly taken @hazelxue claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
skillfs: one malformed chat-log line aborts the entire skill-usage analysis (skill_usage_from_chat_logs.py)Possibly taken @zjncs claimed this today. Opencomponent:skillfs
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
agentic-os-org/ANOLISA#6116 · 1 comment ·
Maintainers usually reply within 1 day