Panic triggerred by the OOM-action output the same panic log with the panic that is unexpected · pingcap/tidb#27089

(5 comments) (0 reactions) (0 assignees)Go (40,090 stars) (6,186 forks)batch import

help wantedsig/executiontype/enhancement

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

 select  count(*) FROM mysql.stats_fm_sketch WHERE table_id = 418;
+----------+
| count(*) |
+----------+
|   143268 |
+----------+
1 row in set (0.06 sec)

[adapter.go:335] ["execute sql panic"] [sql="delete from mysql.stats_fm_sketch where table_id = 418"] [stack="github.com/pingcap/tidb/executor.(*ExecStmt).Exec.func1\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:335\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:965\ngithub.com/pingcap/tidb/util/memory.(*PanicOnExceed).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/util/memory/action.go:128\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/store/copr.(*rateLimitAction).Action\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1111\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/util/memory/tracker.go:284\ngithub.com/pingcap/tidb/executor.(*DeleteExec).removeRow\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/delete.go:211\ngithub.com/pingcap/tidb/executor.(*DeleteExec).deleteOneRow\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/delete.go:62\ngithub.com/pingcap/tidb/executor.(*DeleteExec).deleteSingleTableByChunk\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/delete.go:112\ngithub.com/pingcap/tidb/executor.(*DeleteExec).Next\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/delete.go:50\ngithub.com/pingcap/tidb/executor.Next\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/executor.go:286\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:589\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handlePessimisticDML\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:608\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:468\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:419\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/session/session.go:1635\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/session/session.go:1530\ngithub.com/pingcap/tidb/session.(*session).ExecuteInternal\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/session/session.go:1250\ngithub.com/pingcap/tidb/statistics/handle.(*Handle).DeleteTableStatsFromKV\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/statistics/handle/gc.go:220\ngithub.com/pingcap/tidb/statistics/handle.(*Handle).gcTableStats\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/statistics/handle/gc.go:74\ngithub.com/pingcap/tidb/statistics/handle.(*Handle).GCStats\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/statistics/handle/gc.go:48\ngithub.com/pingcap/tidb/domain.(*Domain).updateStatsWorker\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb/domain/domain.go:1312"]

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

Release Version: v5.1.1
Edition: Community
Git Commit Hash: 797bddd25310ed42f0791c8eccb78be8cce2f502
Git Branch: heads/refs/tags/v5.1.1
UTC Build Time: 2021-07-28 11:00:26
GoVersion: go1.16.4
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false

Contributor guide

Tech stack: gosql
Domain: backenddatabase
Issue type: bug
Difficulty: 4
Estimated time: 3-5 days
Activity status: stale
Clarity: mostly clear
Prerequisites: Go programmingTiDB internalsMemory managementPanic handling
Newbie friendliness: 25
Research direction: Investigate the panic in util/memory/action.go (line 128) and store/copr/coprocessor.go (line 1111) to understand the rateLimitAction and PanicOnExceed interactions. Reproduce the delete operation that triggers the panic, and check if the panic message is duplicated due to the OOM action. Look for related discussions in the issue comments and assess whether a similar fix has been attempted elsewhere in the codebase.