cube-js/cube

Store pre-aggregations in a directory within an export bucket with the BigQuery driver

Open

#4,252 opened on Mar 21, 2022

View on GitHub
 (3 comments) (2 reactions) (0 assignees)Rust (1,965 forks)batch import
driver:bigqueryhelp wanted

Repository metrics

Stars
 (19,563 stars)
PR merge metrics
 (Avg merge 5d 16h) (138 merged PRs in 30d)

Description

Is your feature request related to a problem? Please describe. I want to store pre-aggregations for different tenants in separate folders within the same export bucket.

Describe the solution you'd like Maybe to have CUBEJS_DB_EXPORT_BUCKET changed to CUBEJS_DB_EXPORT_PATH, or at least have the semantic changes to support both plain buckets and s3://... or gs://... paths like boto/gsutil does.

Describe alternatives you've considered There is no alternative solution except for using different buckets altogether.

Additional context The BigQuery driver uses a bucket eg CUBEJS_DB_EXPORT_BUCKET=cube-cloud-staging-export-bucket. The Athena driver supports S3 paths eg CUBEJS_DB_EXPORT_BUCKET=s3://cubejs-benchmarks/home/app/.../.

Contributor guide

Research direction
Investigate the BigQuery driver's handling of the CUBEJS DB EXPORT BUCKET environment variable and propose a change to support bucket paths (e.g., 'gs://bucket/path') similar to how the Athena driver supports S3 paths.
Tech stack
typescriptjavascript
Domain
backendclouddatabase
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GitNode.js
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75