flinkStreamsql+flink1.10启动报错

Open
#314 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
java, sql
Domain
backend, data

Research direction

Reproduce the supplied yarnPer launch with Flink 1.10, then trace executor creation from com.dtstack.flink.sql.Main through ExecuteProcessHelper and StreamTableEnvironmentImpl. Inspect the launcher classpath and planner factory loading around JobGraphBuildUtil and YarnJobClusterExecutor. Done means the executor factory resolves and the SQL job starts without the planner-module error.

Written by the indexing model from the issue text.

Description

log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
------------program params-------------------------
-mode
yarnPer
-name
flink_test
-sql
CREATE+TABLE+MyTable%28%0A%09id+int%2C%0A++++name+varchar%2C%0A++++productID+varchar%0A+%29WITH%28%0A++++type+%3D%27kafka%27%2C%0A++++bootstrapServers+%3D%27datanode-1%3A6667%27%2C%0A++++zookeeperQuorum+%3D%27namenode-1%3A2181%2Fkafka%27%2C%0A++++offsetReset+%3D%27latest%27%2C%0A++++topic+%3D%27ee_test%27%2C%0A++++parallelism+%3D%272%27%0A+%29%3B%0A%0A%0ACREATE+TABLE+MyResult%28%0A++++id+int%2C%0A%09city+varchar%0A+%29WITH%28%0A++++type+%3D%27mysql%27%2C%0A++++url+%3D%27jdbc%3Amysql%3A%2F%2Fdatanode-1%3A3306%2Ftest%3FcharacterEncoding%3Dutf8%26useSSL%3Dtrue%27%2C%0A++++userName+%3D%27root%27%2C%0A++++password+%3D%27root%27%2C%0A++++tableName+%3D%27flink_test1%27%2C%0A++++parallelism+%3D%271%27%0A+%29%3B%0A%0Ainsert%0Ainto%0A++++MyResult%0A++++select%0A+++++id+as+id%2Cname+as+city%0A++++from+%0A++++MyTable%0A
-yarnconf
/usr/hdp/current/hadoop-client/etc/hadoop
-localSqlPluginPath
/opt/flinkStreamSQL/plugins
-remoteSqlPluginPath
/opt/flinkStreamSQL/plugins
-confProp
{}-flinkconf
-flinkJarPath
/opt/flink-1.10/lib
-queue
default
-yarnSessionConf
{}
-pluginLoadMode
CLASSPATH
-logLevel
info

Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not instantiate the executor. Make sure a planner module is on the classpath
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:79)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:101)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:56)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:85)
at com.dtstack.flink.sql.launcher.utils.JobGraphBuildUtil.buildJobGraph(JobGraphBuildUtil.java:80)
at com.dtstack.flink.sql.launcher.executor.YarnJobClusterExecutor.exec(YarnJobClusterExecutor.java:65)
at com.dtstack.flink.sql.launcher.LauncherMain.main(LauncherMain.java:144)
Caused by: org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
at org.apache.flink.table.api.java.internal.StreamTableEnvironmentImpl.lookupExecutor(StreamTableEnvironmentImpl.java:147)
at org.apache.flink.table.api.java.internal.StreamTableEnvironmentImpl.create(StreamTableEnvironmentImpl.java:116)
at org.apache.flink.table.api.java.StreamTableEnvironment.create(StreamTableEnvironment.java:112)
at org.apache.flink.table.api.java.StreamTableEnvironment.create(StreamTableEnvironment.java:83)
at com.dtstack.flink.sql.exec.ExecuteProcessHelper.getStreamExecution(ExecuteProcessHelper.java:149)
at com.dtstack.flink.sql.Main.main(Main.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
... 8 more
Caused by: org.apache.flink.table.api.NoMatchingTableFactoryException: Could not find a suitable table factory for 'org.apache.flink.table.delegation.ExecutorFactory' in
the classpath.

Reason: No factory supports the additional filters.

The following properties are requested:
class-name=org.apache.flink.table.executor.StreamExecutorFactory
streaming-mode=true

The following factories have been considered:
org.apache.flink.table.planner.delegation.BlinkExecutorFactory
at org.apache.flink.table.factories.ComponentFactoryService.find(ComponentFactoryService.java:71)
at org.apache.flink.table.api.java.internal.StreamTableEnvironmentImpl.lookupExecutor(StreamTableEnvironmentImpl.java:138)
... 18 more

Dominant language
Java
Stars
2k
Forks
913
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DTStack/flinkStreamSQL

All issues in DTStack/flinkStreamSQL

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.