Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

[Bug] BE aborts via std::terminate when a VARIANT read exceeds the query memory limit (allocation in noexcept release path)

Ouverte
  • Quelque chose ne va pas ?
#68,898 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
22/100
Type d'issue
Bug
Clarté
Plutôt claire
Activité
Active
Stack technique
cpp
Domaine
database

Piste de recherche

Start at release_batch_container in be/src/core/value/variant/variant_batch_builder.cpp (around line 134) and at _take_encoded_metadata() (around line 342), then check the noexcept functions there that call it, including release_scratch(). Run or write a test that drives the VARIANT V2 assembly path under a tight memory tracker limit. Done means the query returns MEM_LIMIT_EXCEEDED without the BE terminating, and the noexcept release paths no longer allocate.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Search before asking
  • I had searched in the issues and found no similar issues.
Version

4.1.4 (cloud mode, arm64); master is the same. On master release_batch_container is be/src/core/value/variant/variant_batch_builder.cpp:134 and _take_encoded_metadata() is :342.

What's Wrong?

A query that reads a VARIANT V2 column and exceeds its memory limit aborts the BE process with std::terminate, instead of failing the query with MEM_LIMIT_EXCEEDED. When the query runs on every BE, every BE aborts at once. In our case all 4 BEs aborted within about 10 s and restarted.

FE error for the query: [MEM_ALLOC_FAILED]Allocator mem tracker check failed, [MEM_LIMIT_EXCEEDED]failed alloc size 13.68 MB.

Symbolized BE stack:

#0  std::terminate handler
#3  __clang_call_terminate
#4  doris::VariantMetadataBuilder::_take_encoded_metadata()   (allocation, core/custom_allocator.h:133)
#5  doris::VariantBatchBuilder::finish_batch()                (core/value/variant/variant_batch_builder.cpp:1728)
#6  doris::segment_v2::variant_v2::VariantAssembler::assemble(...)
    ... segment iterator / scan operator frames

Cause:

template <typename Container>
void release_batch_container(Container& container) noexcept {
    Container().swap(container);
}

PaddedPODArray<char> VariantMetadataBuilder::_take_encoded_metadata() noexcept {
    ...
    release_batch_container(_impl->keys);   // Keys = std::deque<PaddedPODArray<char>, CustomStdAllocator<...>>
    ...
}

A default-constructed libstdc++ std::deque allocates its map and first node. With CustomStdAllocator, that allocation goes through MemoryAllocator::alloc, which throws doris::Exception (MEM_ALLOC_FAILED, "Allocator mem tracker check failed") once the thread's memory tracker is over its limit. The throw happens inside a noexcept function, so the runtime calls std::terminate. release_scratch() is also noexcept and uses the same helper. Other noexcept functions in this file may allocate as well, and are worth auditing for the same pattern.

What You Expected?

The query fails with MEM_LIMIT_EXCEEDED and the BE keeps running.

How to Reproduce?

A table with a VARIANT column (Variant V2) holding about 1B rows of nested JSON. A query that materializes the VARIANT for many rows under the default query memory limit, e.g.:

SELECT count(*), sum(length(j))
FROM (SELECT CAST(v AS STRING) AS j FROM t
      WHERE ts >= '...' AND ts < '...'
      ORDER BY ts DESC, id LIMIT 1000000) x;

Any query whose VARIANT read pushes the memory tracker over its limit while finish_batch runs should reproduce it. (Separately, the CAST is evaluated in the scan projection for every row before the top-N, rather than being deferred by lazy materialization. That makes this query far more memory-hungry than its LIMIT suggests.)

Anything Else?

Possible fix: make the release path unable to allocate or throw. For example, fall back to clear() when constructing an empty container throws, or drop noexcept where no caller relies on it, so the memory-limit exception reaches the query as a normal error.

Are you willing to submit PR?
  • Yes I am willing to submit a PR!
Code of Conduct
Langage dominant
Java
Étoiles
16.1k
Forks
4k
Merge moyen
2 j 23 h
PR mergées (30 j)
686

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de apache/doris

Toutes les issues de apache/doris

Issues similaires

Plus d'issues Java

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.