apache/pinot

Support query response dump in different formats and location

Open

#6,939 opened on May 18, 2021

View on GitHub
 (3 comments) (2 reactions) (0 assignees)Java (1,234 forks)batch import
featuregood first issue

Repository metrics

Stars
 (4,937 stars)
PR merge metrics
 (Avg merge 6d 7h) (186 merged PRs in 30d)

Description

From the query endpoint, Pinot currently only supports customized JSON format.

It would be great to support:

  1. user can specify the response format, e.g. CSV
  2. user can specify a file to dump the query response to, e.g. local or remote(s3) file or directory.
  3. user can specify the compression algorithm for output file.

This can be implemented in broker query API or we can do all the reformat and response dump using org.apache.pinot.tools.admin.command.PostQueryCommand

Contributor guide

Research direction
Investigate the broker query API and PostQueryCommand in Pinot to understand how response handling is currently implemented, then extend to support CSV format, file output to local or S3, and compression.
Tech stack
java
Domain
backendapi
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
JavaApache Pinot
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40