[Bug] Spark engine fails to open sessions with SERVER_LOCAL share level

Open
  • Is something wrong?
#7,844 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@mangeshraut712 is already working on this.

Since Nov 4, 2026.

  • #7845 by @mangeshraut712 — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
scala, spark
Domain
backend, databases

Research direction

Read externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSQLSessionManager.scala, especially getOrNewSparkSession around line 112, where the issue reports that SERVER_LOCAL is missing from the share-level match. Trace the server-side engine discovery and session routing before changing the branches. Add regression coverage for OpenSession, SQL, and CloseSession across the listed isolation and share-level cases; done means session state and connection behavior match those expectations.

Written by the indexing model from the issue text.

Description

good first issue help wanted kind:bug module:spark
Code of Conduct
Search before asking
  • I have searched in the issues and found no similar issues.
Describe the bug

With kyuubi.engine.share.level=SERVER_LOCAL, the Spark SQL engine fails to open a session with a scala.MatchError, before executing SQL.

We reproduced this through both Binary TLS JDBC and HTTP TLS JDBC with valid ASCII credentials, Spark 3.5.9, Java 17, and spark.master=local[2]. Changing only the share level to SERVER allowed the same connection checks and SQL queries to pass.

Steps to reproduce:

  1. Start a Kyuubi server with a Spark SQL engine and the configuration below.
  2. Open a regular JDBC connection with valid credentials and wait for engine OpenSession.
  3. Observe the exception below. A query such as SELECT 1 cannot run because session creation fails.
  4. Set kyuubi.engine.share.level=SERVER and restart with a fresh engine as a comparison.
Affects Version(s)

master at f3125812e364ad1ad687d2e57b693de114a4991d, using 1.13.0-SNAPSHOT artifacts. The first affected release has not been established.

Kyuubi Server Log Output

The client reports the engine OpenSession exception shown below.

Kyuubi Engine Log Output
Caused by: java.lang.RuntimeException: scala.MatchError:
SERVER_LOCAL (of class scala.Enumeration$Val)
  at SparkSQLSessionManager.getOrNewSparkSession(SparkSQLSessionManager.scala:112)
Kyuubi Server Configurations
kyuubi.engine.type=SPARK_SQL
kyuubi.engine.share.level=SERVER_LOCAL
kyuubi.engine.doAs.enabled=false
kyuubi.engine.single.spark.session=false
kyuubi.engine.user.isolated.spark.session=true
Kyuubi Engine Configurations
spark.master=local[2]
Additional context

Contributions are welcome.

In SparkSQLSessionManager.getOrNewSparkSession, the share-level match handles CONNECTION, USER, GROUP, and SERVER, but omits SERVER_LOCAL.

Consider handling SERVER_LOCAL alongside SERVER in both the guarded per-connection branch and the per-user cache branch. Preserve the server-side engine discovery/routing boundary: SERVER_LOCAL shares an engine within the server that launched it.

Suggested regression coverage:

Please verify through OpenSession, SQL, and CloseSession:

  • With single-session mode disabled and user isolation enabled, two connections for the same user have independent temporary views and SQLConf.
  • With user isolation disabled, the same user reuses session state, different users remain isolated, and reference counting and idle eviction preserve active sessions.
  • Closing one connection leaves other connections usable.
  • Explicit single-session mode and existing share levels retain their behavior.
Are you willing to submit PR?
  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
  • No. I cannot submit a PR at this time.
Dominant language
Scala
Stars
2.4k
Forks
1k
Avg merge
18h 17m
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/kyuubi

All issues in apache/kyuubi

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.