Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Optimize TsFileWriter::do_check_schema CPU overhead for repeated wide-tablet writes

Open
  • Is something wrong?
#885 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@kkzi is already working on this.

Since Sep 2, 2026.

  • #934 by @kkzi — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
cpp
Domain
data, performance

Research direction

Start at TsFileWriter::write_tablet() and do_check_schema, tracing the device and measurement-schema lookups and the chunk_writers and data_types resolution described in the issue. Compare repeated identical-device writes with changing-device or dynamic-schema writes; done means reducing repeated lookup overhead while preserving existing validation behavior.

Written by the indexing model from the issue text.

Description

performance

Problem

TsFileWriter::do_check_schema shows noticeable CPU overhead when writing repeated tablets with the same device and schema.

In our workload, each writer repeatedly writes tablets for a fixed device with a fixed set of measurements. However,
write_tablet() calls do_check_schema() for every tablet write. This appears to repeatedly perform schema lookup and
measurement-name matching even though the schema has already been registered and does not change during the file lifecycle.

Evidence

Using Windows Performance Analyzer on a 60-second TsFile archive workload, the active processing window showed:

  • TsFileArchive.dll!storage::TsFileWriter::do_check_schema<MeasurementNamesFromTablet> as the top TsFileArchive function
    hotspot
  • std::string::compare also appeared as a related hotspot
  • The workload writes tablets with the same device and the same schema repeatedly
  • TsFile writing took longer than the input data duration

Example hotspot:

TsFileArchive.dll!storage::TsFileWriter::do_check_schema<storage::MeasurementNamesFromTablet>

Suspected Cause

write_tablet() creates or looks up schema information on every call:

  • creates a StringArrayDeviceID from tablet.insert_target_name_
  • looks up the device in schemas_
  • iterates all measurement names
  • looks up each measurement in measurement_schema_map_
  • fills chunk_writers and data_types

For wide tablets and high-frequency writes, this repeated per-tablet schema validation becomes expensive.

Suggested Optimization

Add a cached or prepared schema path for repeated tablet writes.

Possible approaches:

  1. Cache the resolved schema for the last used device/tablet schema inside TsFileWriter.
  2. Add an explicit prepared API, for example:

PreparedTabletSchema prepare_tablet_schema(device_id, schema_vec);
int write_tablet_prepared(const Tablet& tablet, const PreparedTabletSchema& prepared);

The cached/prepared data could include:

  • MeasurementSchemaGroup*
  • resolved ChunkWriter* list
  • resolved TSDataType list

This would allow repeated writes with the same device and schema to skip per-column name lookup.

Expected Benefit

Reduce CPU overhead in high-throughput TsFile writing workloads, especially for wide schemas where the same tablet schema is
written repeatedly.

Notes

This should preserve the existing validation behavior for dynamic schemas or changing devices. The optimization can be limited
to cache hits where both device and schema identity match.

Dominant language
Java
Stars
206
Forks
105
Avg merge
22h 54m
Merged PRs (30d)
27

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/tsfile

All issues in apache/tsfile

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.