delta-io/delta

[Feature Request][Kernel] Make kernel benchmark configurable using command line arguments

Open

#5420 opened on Oct 30, 2025

View on GitHub
 (2 comments) (0 reactions) (1 assignee)Scala (8,807 stars) (2,100 forks)batch import
delta-kernelenhancementgood first issue

Description

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Overview

Motivation

The kernel benchmarking framework currently hard codes benchmark parameters. There are multiple ways users may want to augment a benchmark run including:

  • Filtering benchmark specs by name
  • Configure the number of warmup iterations
  • Configure the number of measurement iterations
  • Configure the warmup time.

Further details

Create a command line argument parser that accepts the above options as configurable parameters. Use the current parameters as the defaults for the benchmark.

For filtering benchmark specs, perform a substring match over each variant's full name. --include-test read_metadata For all other paramaeters, simply take an integer --warum-iterations 10

You can find the relevant file in kernel/kernel-defaults/src/test/java/io/delta/kernel/defaults/benchmarks/WorkloadBenchmark.java`

Contributor guide

Tech stack
java
Domain
backendperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
blocked
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
JavaCommand line argument parsingDelta Kernel benchmarks
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Start by examining the file `kernel/kernel defaults/src/test/java/io/delta/kernel/defaults/benchmarks/WorkloadBenchmark.java` to understand the current hardcoded benchmark parameters. The task is to add a command line argument parser (e.g., using picocli or Apache Commons CLI) that accepts ` include test`, ` warmup iterations`, ` measurement iterations`, and ` warmup time` as options, while keeping the existing values as defaults. Look at how the benchmark is currently invoked and ensure the new CLI supports substring matching for test names and integer values for other parameters. Consider any existing test infrastructure or build configuration that might need updates.