parquet-cli rewrite option

Open

#2,981 opened on Aug 7, 2024 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Feature
Clarity: Mostly clear
Activity status: Stale
Tech stack: google-cloud, java
Domain: cli, cloud

Research direction

Start with the parquet-cli rewrite command and reproduce the reported gs:// input and output failure. Trace how rewrite resolves filesystem schemes, then check the Hadoop filesystem setup used by the CLI. Done means the requested GCS prune-columns workflow is supported or its limitation is clearly documented, with the error behavior covered by a test if the project provides one.

Written by the indexing model from the issue text.

Description

Describe the usage question you have. Please include as many useful details as possible.

Hi ,

is it possible to read directly from a gcs bucket to prune a column
like rewrite -i gs:/sourcebbucket/part-00549.parquet -o gs://targetbucket/newdata/dd --prune-columns col4

i am getting error
java.lang.RuntimeException: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "gs"

Component(s)

No response

Dominant language: Java
Stars: 3.1k
Forks: 1.6k
Avg merge: 6d 16h
Merged PRs (30d): 36

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

chore: aspectjweaver(EPL-2.0)와 AGPL-3.0 호환성 정리 Open

documentation

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

inu-appcenter/memorIN-backend#288 · Opened Sep 24, 2026
Vulnerability Nessus 12085 - Apache Tomcat Default Files (with killbill/killbill:0.24.21) Open

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

killbill/killbill#2314 · 1 comment · Opened Sep 24, 2026
Work list, Cases and Compliance overlap, and nothing says which one to use Open

frontend maui-pilot pilot-ask question

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Taleef7/workwell#698 · Opened Sep 24, 2026
[Bug][UI] Replace the status of the button in the Sessions Tab Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Netcracker/qubership-integration-platform#918 · Opened Sep 24, 2026
executions.Query — startDate and timeRange filters are sent with inverted comparison operators Open

area/plugin

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

kestra-io/plugin-kestra#190 · Opened Sep 24, 2026

More Java issues