cube-js/cube

How one can disable cache or preaggregation for large datasets?

Open

#5,341 opened on Sep 22, 2022

View on GitHub
 (2 comments) (2 reactions) (0 assignees)Rust (1,965 forks)batch import
help wanted

Repository metrics

Stars
 (19,563 stars)
PR merge metrics
 (Avg merge 5d 16h) (138 merged PRs in 30d)

Description

Problem

Sometimes users can run smth like 'Select many dimensions and metrics for long period ". When it happens, huge portion of data is added to memory cache (we are still using redis, but I think problem will remain after adding cube store).

How can we disable adding result of query to cache by its size (e.g. limit 30 mb)? For us it's cheaper to make roundrip to db)

Contributor guide

Research direction
Investigate Cube.js caching mechanisms and configure a size limit or disable cache for large queries. Consider using environment variables or query level settings.
Tech stack
nodejsredis
Domain
backenddatabase
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Node.jsRedis
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50