serializeToBundle issue

Open
  • Is something wrong?
#35 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, spark
Domain
machine-learning

Research direction

Reproduce the Python example with Spark 3.3.0 and MLeap 0.22.0, then inspect SimpleSparkSerializer.scala and SparkBundleContext.scala from the stack trace. Determine the supported Spark-version check and verify the relevant compatibility combinations. Done means the failure is resolved for the reported setup or the exact incompatibility and supported versions are documented.

Written by the indexing model from the issue text.

Description

Hi,
I am trying to serialize a simple spark model using mleap serializeToBundle method, but for some reason its failing with the below attached error, that the spark version is not supported. I have tried all the combinations of mleap and spark version as mentioned in the mleap doc, but the error remains the same.

Any help/suggestions on the resolving this would be appreciated.

python: 3.8
spark version 3.3.0
mleap: 0.22.0

data = spark.read.format("libsvm").load("sample_libsvm_data.txt")
featureIndexer = VectorIndexer(inputCol="features", outputCol="indexedFeatures", maxCategories=4).fit(data)
(trainingData, testData) = data.randomSplit([0.7, 0.3])
dt = DecisionTreeRegressor(featuresCol="indexedFeatures")
pipeline = Pipeline(stages=[featureIndexer, dt])
model = pipeline.fit(trainingData)

model.serializeToBundle("jar:file:/pyspark.example.zip", model.transform(data))

2024-07-22 17:29:21.070 IST
: java.lang.ExceptionInInitializerError
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer$$anonfun$1.apply(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer$$anonfun$1.apply(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at scala.Option.map(Option.scala:230)
2024-07-22 17:29:21.070 IST
	at ml.combust.mleap.spark.SimpleSparkSerializer.serializeToBundle(SimpleSparkSerializer.scala:17)
2024-07-22 17:29:21.070 IST
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-07-22 17:29:21.070 IST
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2024-07-22 17:29:21.070 IST
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-07-22 17:29:21.070 IST
	at java.lang.reflect.Method.invoke(Method.java:498)
2024-07-22 17:29:21.070 IST
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2024-07-22 17:29:21.070 IST
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2024-07-22 17:29:21.070 IST
	at py4j.Gateway.invoke(Gateway.java:282)
2024-07-22 17:29:21.070 IST
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2024-07-22 17:29:21.070 IST
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
2024-07-22 17:29:21.070 IST
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
2024-07-22 17:29:21.070 IST
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
2024-07-22 17:29:21.070 IST
	at java.lang.Thread.run(Thread.java:750)
2024-07-22 17:29:21.070 IST
Caused by: java.lang.IllegalStateException: unsupported Spark version: 3.3.0
	at org.apache.spark.ml.bundle.SparkBundleContext$.<init>(SparkBundleContext.scala:23)
	at org.apache.spark.ml.bundle.SparkBundleContext$.<clinit>(SparkBundleContext.scala)
	... 16 more
Dominant language
Shell
Stars
15
Forks
21
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from combust/mleap-docs

All issues in combust/mleap-docs

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.