Field caps request returns too much data

Open
#172 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
typescript
Domain
api

Research direction

Start by reviewing issue #20 and the datasource entry point that issues _field_caps requests for autocomplete. Clarify whether the intended result is per-datasource disabling or selecting fields/index sources, then define completion as preventing excessive dynamic-field responses without blocking log requests.

Written by the indexing model from the issue text.

Description

After changes introduced in https://github.com/quickwit-oss/quickwit-datasource/issues/20, when w're trying to request logs from otel indices for a large period eg 30 days, request for _field_caps either takes a lot of time, holding logs request or even fails due to its response size, because attributes.* field may return infinite amount of fields due to it's highly dynamic nature. I believe it would be nice to have ability to either disable the autocomplete feature for specific datasource or have ability to pick fields/index source for autocomplete to make better experience here.

Dominant language
TypeScript
Stars
76
Forks
44
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from quickwit-oss/quickwit-datasource

All issues in quickwit-oss/quickwit-datasource

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.