MPP query fails with "Cannot LZ4_decompress_safe" when one probe row matches many build rows
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 48/100
Direzione di ricerca
Start with JoinPartition.cpp at Adder<KIND, Strictness::All, Map>::addFound, then trace how HashPartitionWriter flushes partitions and how CompressionCodecLZ4::doCompressData handles oversized input. Reproduce with the SQL query in the report using legacy hash join and FAST exchange compression. Done means the query returns the expected count and maximum value without decompression or packet-size errors.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Bug Report
1. Minimal reproduce step (Required)
Use a cluster with 3 TiFlash nodes.
CREATE TABLE a (k BIGINT NOT NULL, c BIGINT NOT NULL);
CREATE TABLE b (k BIGINT NOT NULL, s VARCHAR(64) NOT NULL);
CREATE TABLE c (c BIGINT NOT NULL);
INSERT INTO a VALUES (1, 3);
INSERT INTO c VALUES (3);
SET SESSION cte_max_recursion_depth = 10000000;
INSERT INTO b WITH RECURSIVE g(n) AS (SELECT 1 UNION ALL SELECT n + 1 FROM g WHERE n < 1000000)
SELECT 1, LPAD(n, 30, 'x') FROM g;
SET SESSION tidb_dml_type = 'bulk';
INSERT INTO b SELECT * FROM b; -- Do this 6 times. Then b has 64,000,000 rows.
ALTER TABLE a SET TIFLASH REPLICA 1;
ALTER TABLE b SET TIFLASH REPLICA 1;
ALTER TABLE c SET TIFLASH REPLICA 1;
-- Wait until the TiFlash replicas are available.
ANALYZE TABLE a, b, c;
SET SESSION tidb_isolation_read_engines = 'tiflash,tidb';
SET SESSION tidb_enforce_mpp = ON;
SET SESSION tidb_broadcast_join_threshold_count = 0;
SET SESSION tidb_broadcast_join_threshold_size = 0;
SET SESSION tiflash_hash_join_version = 'legacy';
SET SESSION mpp_exchange_compression_mode = 'FAST';
SELECT /*+ HASH_JOIN_BUILD(b) */ STRAIGHT_JOIN count(*), max(b.s)
FROM a JOIN b ON a.k = b.k JOIN c ON a.c = c.c;
The first join builds on b and probes the one row of a. An ExchangeSender sends the join output to the second
join by a hash partition on a.c. If the error is Packet is too large to send, the partition stays on the same
node. Then use another value for a.c and c.c, for example 2.
2. What did you expect to see? (Required)
The query returns 64000000 and the largest value of b.s.
3. What did you see instead (Required)
The query fails after 11 s:
ERROR 1105 (HY000): other error for mpp stream: Code: 271, e.displayText() = DB::Exception: Cannot LZ4_decompress_safe: ... ExchangeReceiver_42
| Setting | Result |
|---|---|
FAST, the partition goes to another node |
Fails: Cannot LZ4_decompress_safe (Code 271) |
NONE, or the partition stays on the same node |
Fails: Packet is too large to send, size : 3520000195 (Code 49) |
tiflash_hash_join_version = 'optimized' |
Succeeds in 6–8 s |
The packet has 64,000,000 rows × 55 bytes + 195 bytes. Thus, all output rows of the join go into one packet.
The same failure occurs in JCC-H q7 at scale factor 1,000 (TiFlash mode). The join lineitem JOIN orders builds on
lineitem, and one row of orders matches 75,200,000 rows. The packets have 4,211,323,072 and 8,874,106,922 bytes.
Cause:
Adder<KIND, Strictness::All, Map>::addFound(JoinPartition.cpp) checksmax_block_sizeonly when the output
block already has rows. Thus, the first probe row of a block adds all its matches to the block.HashPartitionWriterusesmax_buffered_rowsandmax_buffered_bytesonly to start a flush. A flush writes all
buffered rows of a partition as one packet. All rows of this block have the same partition key.CompressionCodecLZ4::doCompressDatadoes not check the result ofLZ4_compress_fast. For an input larger than
LZ4_MAX_INPUT_SIZE(2,113,929,216 bytes), LZ4 returns 0, and the receiver fails inLZ4_decompress_safe.
CompressionCodecLZ4HCchecks this result. Without compression,checkPacketSize(limit 2^31 bytes) gives the
real cause.
Hash join v2 stops each output block at max_block_size, also inside the matches of one probe row. But the default
of tiflash_hash_join_version is legacy.
4. What is your TiFlash version? (Required)
v9.0.0-beta.2.pre-237-g969db6452 (Git Commit Hash: 969db6452afbf0ffde65013520f01d8e4fac917d), with TiDB f40ad1d845.
The code of the cause is the same on master f3b7883955 (2026-10-28).
- Lingua principale
- C++
- Stelle
- 1k
- Fork
- 428
- Merge medio
- 2g 7h
- PR unite (30g)
- 100
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Nessuna guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di pingcap/tiflash
-
component/compute type/bug
Difficoltà 2/5 1-2 giorni Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
Out-of-bounds read in MyDateTime formatting from unvalidated packed valuesForse già presa @ChangRui-Ryan l’ha presa oggi. Apertaaffects-25.10 affects-26.3 affects-26.9 affects-7.5 affects-8.1 affects-8.5 severity/major type/bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 30/100
I maintainer di solito rispondono entro 1 giorno
-
component/storage severity/moderate type/bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
pingcap/tiflash#11335 · 5 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Missing validation in JSON constructors, LEFT expressions, and CH-block decodingForse già presa @windtalker l’ha presa oggi. Apertatype/bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
I maintainer di solito rispondono entro 1 giorno
-
[disagg] Query fails with Code: 11000 NoSuchKey when S3 GC (ScanThenDelete) removes a DMFile that a compute node still readsForse già presa @JinheLin l’ha presa 6 giorni fa. Apertamay-affects-25.10 may-affects-26.3 may-affects-26.9 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/moderate type/bug
pingcap/tiflash#11291 · 1 commento · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di pingcap/tiflash
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
useless statementAperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 86/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
grizzlypeak3d/DJV#962 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno