microsoft/SynapseML

NullPointerException while compute accuracy with ComputeModelStatistics

Open

#736 aperta il 13 nov 2019

Vedi su GitHub
 (7 commenti) (0 reazioni) (1 assegnatario)Scala (861 fork)batch import
area/documentationbuggood first issuehelp wanted

Metriche repository

Star
 (5228 star)
Metriche merge PR
 (Merge medio 23h 16m) (2 PR mergiate in 30 g)

Descrizione

Version

com.microsoft.ml.spark:mmlspark_2.11:jar:0.18.1
spark= 2.4.3
scala=2.11.12

data (csv with header) https://gist.github.com/ttpro1995/69051647a256af912803c9a16040f43a

download data and save as csv file, put into folder /data/public/HIGGS/higgs.test.predictioncsv

val data = spark.read.option("header","true").option("inferSchema", "true").csv("/data/public/HIGGS/higgs.test.predictioncsv")

Schema

root
 |-- label: double (nullable = true)
 |-- prediction: double (nullable = true)

Code

import com.microsoft.ml.spark.train.ComputeModelStatistics
val metricsCompute = new ComputeModelStatistics().setLabelCol("label").setScoresCol("prediction").setEvaluationMetric("accuracy")

val result_metrics = metricsCompute.transform(data)

Exception

java.lang.NullPointerException
  at org.apache.spark.sql.Column.<init>(Column.scala:135)
  at org.apache.spark.sql.Column$.apply(Column.scala:38)
  at org.apache.spark.sql.functions$.col(functions.scala:90)
  at com.microsoft.ml.spark.train.ComputeModelStatistics.selectAndCastToDF(ComputeModelStatistics.scala:256)
  at com.microsoft.ml.spark.train.ComputeModelStatistics.selectAndCastToRDD(ComputeModelStatistics.scala:265)
  at com.microsoft.ml.spark.train.ComputeModelStatistics.predictionAndLabels$lzycompute$1(ComputeModelStatistics.scala:99)
  at com.microsoft.ml.spark.train.ComputeModelStatistics.predictionAndLabels$1(ComputeModelStatistics.scala:95)
  at com.microsoft.ml.spark.train.ComputeModelStatistics.transform(ComputeModelStatistics.scala:124)
  ... 47 elided

Guida contributor