[VL] Segmentation fault when dumping task for micro benchmark

Aperta
#13,225 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
15/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
cpp, java, scala

Direzione di ricerca

Start from the native frame AlignedBuffer::freeToPool() in libvelox.so, reached through ColumnarBatchOutIterator.nativeClose, VeloxIteratorApi and IteratorsV1.PayloadCloser, which is where the close path runs during task teardown. The issue has no reproduction query or Gluten version, so first reproduce a scan + filter + project task and confirm the SIGSEGV. Done means the crash no longer occurs under that query and the root cause in the buffer lifetime is identified.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug triage
Backend

VL (Velox)

Bug description

Encountered SIGSEGV when dumping a simple scan + filter + project task

Current thread (0x00007f341f012000):  JavaThread "Executor task launch worker for task 22.0 in stage 36.0 (TID 1097)" daemon [_thread_in_native, id=2183, stack(0x00007f33cb4fe000,0x00007f33cb5fe000)]

Stack: [0x00007f33cb4fe000,0x00007f33cb5fe000],  sp=0x00007f33cb5f9a20,  free space=1006k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libvelox.so+0x2393cef]  facebook::velox::AlignedBuffer::freeToPool()+0x1f

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 13266  org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeClose(J)V (0 bytes) @ 0x00007f349de82060 [0x00007f349de82020+0x0000000000000040]
J 19654 c1 org.apache.gluten.backendsapi.velox.VeloxIteratorApi$$Lambda$2259+0x00007f3440c2f198.apply$mcV$sp()V (16 bytes) @ 0x00007f3495f9ea14 [0x00007f3495f9e820+0x00000000000001f4]
J 20304 c2 org.apache.gluten.iterator.IteratorsV1$PayloadCloser.hasNext()Z (14 bytes) @ 0x00007f349e313f50 [0x00007f349e313980+0x00000000000005d0]
J 17252 c2 org.apache.gluten.iterator.IteratorsV1$LifeTimeAccumulator.hasNext()Z (23 bytes) @ 0x00007f349e166a0c [0x00007f349e1669c0+0x000000000000004c]
J 19477 c1 org.apache.gluten.execution.VeloxColumnarToRowExec$$anon$1.hasNext()Z (10 bytes) @ 0x00007f3497818324 [0x00007f34978182c0+0x0000000000000064]
J 16654 c2 scala.collection.Iterator$$anon$10.hasNext()Z (10 bytes) @ 0x00007f349e0bcde8 [0x00007f349e0bcda0+0x0000000000000048]
J 18928 c1 scala.collection.TraversableOnce$FlattenOps$$anon$2.hasNext()Z (52 bytes) @ 0x00007f34976e18d4 [0x00007f34976e17a0+0x0000000000000134]
J 20303 c2 org.apache.gluten.iterator.IteratorsV1$IteratorCompleter.hasNext()Z (23 bytes) @ 0x00007f349e312e08 [0x00007f349e312d40+0x00000000000000c8]
J 21527 c1 org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(ZILscala/collection/Iterator;)Lscala/collection/Iterator; (311 bytes) @ 0x00007f34978eb2ec [0x00007f34978eae00+0x00000000000004ec]
J 21535 c1 org.apache.spark.sql.execution.SparkPlan$$Lambda$3587+0x00007f34410cd9f8.apply(Ljava/lang/Object;)Ljava/lang/Object; (16 bytes) @ 0x00007f34978f4074 [0x00007f34978f3fc0+0x00000000000000b4]
J 19406 c1 org.apache.spark.rdd.RDD$$Lambda$1060+0x00007f34406b5f38.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x00007f34977f6204 [0x00007f34977f60c0+0x0000000000000144]
J 19492 c1 org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (36 bytes) @ 0x00007f349781cc94 [0x00007f349781bb40+0x0000000000001154]
J 18631 c1 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (44 bytes) @ 0x00007f349762647c [0x00007f34976248a0+0x0000000000001bdc]
J 19269 c1 org.apache.spark.scheduler.ResultTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object; (210 bytes) @ 0x00007f34977977ac [0x00007f3497796ec0+0x00000000000008ec]
J 21624 c1 org.apache.spark.TaskContext.runTaskWithListeners(Lorg/apache/spark/scheduler/Task;)Ljava/lang/Object; (74 bytes) @ 0x00007f349792f294 [0x00007f349792f220+0x0000000000000074]
J 20347 c1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;ILscala/collection/immutable/Map;Lscala/Option;)Ljava/lang/Object; (356 bytes) @ 0x00007f3496878d04 [0x00007f3496875f60+0x0000000000002da4]
J 20411 c1 org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Lorg/apache/spark/executor/Executor$TaskRunner;Lscala/runtime/BooleanRef;)Ljava/lang/Object; (54 bytes) @ 0x00007f3496f328ac [0x00007f3496f32660+0x000000000000024c]
J 20410 c1 org.apache.spark.executor.Executor$TaskRunner$$Lambda$892+0x00007f344059b260.apply()Ljava/lang/Object; (12 bytes) @ 0x00007f34971e1344 [0x00007f34971e12c0+0x0000000000000084]
J 19717 c1 org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (211 bytes) @ 0x00007f3496376f8c [0x00007f3496376f20+0x000000000000006c]
J 20328 c1 org.apache.spark.executor.Executor$TaskRunner.run()V (2830 bytes) @ 0x00007f3497870f7c [0x00007f349786c2e0+0x0000000000004c9c]
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+92 [email protected]
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 [email protected]
j  java.lang.Thread.run()V+11 [email protected]
v  ~StubRoutines::call_stub
Gluten version

No response

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Lingua principale
Scala
Stelle
1.6k
Fork
685
Merge medio
4g 16h
PR unite (30g)
84

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/gluten

Tutte le issue di apache/gluten

Issue simili

Altre issue su Scala

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.