I am having a problem running a prediction using a saved MultiLayerPerceptronClassifier model.
JavaScript
x
7
1
# reading the saved model
2
# spark version: version 3.1.2, python3.6
3
from pyspark.ml import PipelineModel
4
from pyspark.ml import Pipeline
5
saved_model = "/home/user/Desktop/algorithms/mlpc_model_8979"
6
read_model = PipelineModel.load(saved_model)
7
JavaScript
1
3
1
# predictions using the read model
2
pred = read_model.transform(df)
3
It throws error:
JavaScript
1
3
1
Py4JJavaError: An error occurred while calling o98.transform.
2
: java.util.NoSuchElementException: Failed to find a default value for layers
3
The original mlpc in the pipeline had layers defined:
JavaScript
1
5
1
mlpc = MultilayerPerceptronClassifier(layers= [200, 30, 10],
2
seed=1234,
3
featuresCol="features",
4
labelCols="label")
5
My attempts to solve it: If I run the pipeline model and do predictions without first saving the model. I works with no error. But saving and re-using the model throws this error.
Any help on how to solve this “Failed to find a default value for layers
” error?
Advertisement
Answer
After a lot of search I discovered a very strange solution.
Had to remove the space before the list = []
in the layers
definition.
JavaScript
1
8
1
|
2
|
3
v
4
mlpc = MultilayerPerceptronClassifier(layers=[200, 30, 10],
5
seed=1234,
6
featuresCol="features",
7
labelCols="label")
8