[Question] spark引擎任务执行失败
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- backend, distributed-systems
Research direction
Reproduce the failure with the shown linkis-cli command and inspect the engine startup path named in the trace, including BDPConfiguration.scala, Logging.scala, and EngineConnServer.scala. Check the reported SLF4J initialization and NoClassDefFoundError sequence first; done means a Spark SQL submission starts successfully without the engine initialization error.
Written by the indexing model from the issue text.
Description
Before asking
- I have read the How to ask for help (我已阅读如何寻求帮助指引文档).
Your environment
- Linkis version used: 1.6.0
- Environment name and version:
- cdh-5.14.2
- hdp-3.1.5
- hive-3.1.3
- spark-3.2.1
- scala-2.12.20
- jdk 1.8.0_121
- ....
Describe your questions
执行 sh bin/linkis-cli -submitUser root -engineType spark-3.2.1 -codeType sql -code "show tables" 出现如下错误：
2025-11-06 17:53:01.053 ERROR jobRequest(LINKISCLI_root_spark_0) execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc: master:9101_0 Failed to async get EngineNode AMErrorException: errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn, master:40416, 0) ticketID: 20ce7bdc-28ca-481c-9eea-1af44d6c4b23 Failed to initialize engine, reason: Failed to start EngineConn, reason: Unexpected problem occured during version sanity check
Reported exception:
java.lang.NullPointerException
at org.slf4j.LoggerFactory.versionSanityCheck(LoggerFactory.java:272)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:126)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.linkis.common.utils.Logging.logger(Logging.scala:28)
at org.apache.linkis.common.utils.Logging.logger$(Logging.scala:28)
at org.apache.linkis.common.conf.BDPConfiguration$.logger$lzycompute(BDPConfiguration.scala:33)
at org.apache.linkis.common.conf.BDPConfiguration$.logger(BDPConfiguration.scala:33)
at org.apache.linkis.common.conf.BDPConfiguration$.init(BDPConfiguration.scala:56)
at org.apache.linkis.common.conf.BDPConfiguration$.$anonfun$new$1(BDPConfiguration.scala:136)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:51)
at org.apache.linkis.common.conf.BDPConfiguration$.(BDPConfiguration.scala:137)
at org.apache.linkis.common.conf.BDPConfiguration$.(BDPConfiguration.scala)
at org.apache.linkis.common.conf.CommonVars.(CommonVars.scala:25)
at org.apache.linkis.common.conf.CommonVars$.apply(CommonVars.scala:58)
at org.apache.linkis.engineconn.executor.conf.EngineConnExecutorConfiguration$.(EngineConnExecutorConfiguration.scala:24)
at org.apache.linkis.engineconn.executor.conf.EngineConnExecutorConfiguration$.(EngineConnExecutorConfiguration.scala)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext.(ExecutorListenerBusContext.scala:26)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext$.(ExecutorListenerBusContext.scala:44)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext$.(ExecutorListenerBusContext.scala)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext.getExecutorListenerBusContext(ExecutorListenerBusContext.scala)
at org.apache.linkis.engineconn.acessible.executor.log.SendAppender.(SendAppender.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:138)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1122)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1047)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1039)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:651)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:249)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:295)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:621)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:694)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:711)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:155)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:309)
at org.apache.log4j.Hierarchy$PrivateLogManager.getContext(Hierarchy.java:85)
at org.apache.log4j.Hierarchy.getContext(Hierarchy.java:98)
at org.apache.log4j.Category.(Category.java:186)
at org.apache.log4j.Logger.(Logger.java:57)
at org.apache.log4j.spi.RootLogger.(RootLogger.java:39)
at org.apache.log4j.LogManager.(LogManager.java:72)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.apache.linkis.common.conf.BDPConfiguration$
at org.apache.linkis.common.conf.CommonVars.(CommonVars.scala:25)
at org.apache.linkis.common.conf.CommonVars$.apply(CommonVars.scala:58)
at org.apache.linkis.common.conf.CommonVars.apply(CommonVars.scala)
at org.apache.linkis.manager.label.conf.LabelCommonConfig.(LabelCommonConfig.java:25)
at org.apache.linkis.manager.label.builder.factory.LabelBuilderFactoryContext.getLabelBuilderFactory(LabelBuilderFactoryContext.java:48)
at org.apache.linkis.engineconn.launch.EngineConnServer$.(EngineConnServer.scala:63)
at org.apache.linkis.engineconn.launch.EngineConnServer$.(EngineConnServer.scala)
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NullPointerException [in thread "main"]
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.linkis.common.utils.Logging.logger(Logging.scala:28)
at org.apache.linkis.common.utils.Logging.logger$(Logging.scala:28)
at org.apache.linkis.common.conf.BDPConfiguration$.logger$lzycompute(BDPConfiguration.scala:33)
at org.apache.linkis.common.conf.BDPConfiguration$.logger(BDPConfiguration.scala:33)
at org.apache.linkis.common.conf.BDPConfiguration$.$anonfun$new$2(BDPConfiguration.scala:138)
at org.apache.linkis.common.conf.BDPConfiguration$.$anonfun$new$2$adapted(BDPConfiguration.scala:137)
at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:71)
at org.apache.linkis.common.conf.BDPConfiguration$.(BDPConfiguration.scala:137)
at org.apache.linkis.common.conf.BDPConfiguration$.(BDPConfiguration.scala)
at org.apache.linkis.common.conf.CommonVars.(CommonVars.scala:25)
at org.apache.linkis.common.conf.CommonVars$.apply(CommonVars.scala:58)
at org.apache.linkis.engineconn.executor.conf.EngineConnExecutorConfiguration$.(EngineConnExecutorConfiguration.scala:24)
at org.apache.linkis.engineconn.executor.conf.EngineConnExecutorConfiguration$.(EngineConnExecutorConfiguration.scala)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext.(ExecutorListenerBusContext.scala:26)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext$.(ExecutorListenerBusContext.scala:44)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext$.(ExecutorListenerBusContext.scala)
at org.apache.linkis.engineconn.executor.listener.ExecutorListenerBusContext.getExecutorListenerBusContext(ExecutorListenerBusContext.scala)
at org.apache.linkis.engineconn.acessible.executor.log.SendAppender.(SendAppender.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:138)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1122)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1047)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1039)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:651)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:249)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:295)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:621)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:694)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:711)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:155)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:309)
at org.apache.log4j.Hierarchy$PrivateLogManager.getContext(Hierarchy.java:85)
at org.apache.log4j.Hierarchy.getContext(Hierarchy.java:98)
at org.apache.log4j.Category.(Category.java:186)
at org.apache.log4j.Logger.(Logger.java:57)
at org.apache.log4j.spi.RootLogger.(RootLogger.java:39)
at org.apache.log4j.LogManager.(LogManager.java:72)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
... 3 more
You can go to this path(/appcom/tmp/root/20251106/spark/20ce7bdc-28ca-481c-9eea-1af44d6c4b23/logs) to find the reason or ask the administrator for help ,ip: master ,port: 9101 ,serviceKind: linkis-cg-linkismanager ,ip: master ,port: 9104 ,serviceKind: linkis-cg-entrance
Eureka service list
eg:
Some logs info or acctch file
linkis-xxx.log:
<!--日志文字贴到这里-->
log file:
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/linkis
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 86/100
-
Scala Compilation is broken on master branch and recent PRsPossibly taken @pjfanning claimed this 13 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
[bug][CGS][entrance] parallelgroup cache not updated when entrance instance goes offlinePossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Difficulty 4/5 3-5 days Newbie friendliness 28/100
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Similar issues
-
Clarify Javadoc for Logger methods taking Object... arguments with regards to Throwable detectionOpen
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
-
[Bug] The shared instance selector's placeholder and no-match text ignore the display languagePossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
apache/rocketmq-dashboard#5561 ·
Maintainers usually reply within 3 days
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HMCL-dev/HMCL#6934 · 1 comment ·
Maintainers usually reply within 1 day
-
test(setup): GitHub configuration tests fail when the temp path is long enough for YAML foldingPossibly taken A pull request linked to this issue is open or already merged. Openbug good first issue help wanted priority medium size S
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
martin-francois/symphony-trello#776 · 1 comment ·
Maintainers usually reply within 1 day