AttributeError: 'DataFrame' object has no attribute 'copy'

I m using autoML(FLAML) with Spark on large data. The error image is given below

train = spark.read.parquet("./train.parquet")
test = spark.read.parquet("./test.parquet")

input_cols = [c for c in train.columns if c != 'target']
vectorAssembler = VectorAssembler(inputCols = input_cols, outputCol = 'features')
vectorAssembler.setHandleInvalid("skip").transform(train).show
train_sprk = vectorAssembler.transform(train)
test_sprk = vectorAssembler.transform(test)

from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification
y = train_sprk["target"]
X = train_sprk[input_cols]
X, y = make_classification()
X_train, X_test, y_train, y_test = train_test_split(X, y)

from flaml import AutoML

automl = AutoML()

from flaml import logger
import logging
logger.setLevel(logging.WARNING)

settings = {
    "time_budget": 200,  # total running time in seconds
    "metric": 'roc_auc',  # can be: 'r2', 'rmse', 'mae', 'mse', 'accuracy', 'roc_auc', 'roc_auc_ovr',
    "estimator_list": ['lgbm', 'xgboost'],                     # 'roc_auc_ovo', 'log_loss', 'mape', 'f1', 'ap', 'ndcg', 'micro_f1', 'macro_f1'
    "task": 'classification',  # task type
    "log_file_name": 'airlines_experiment.log',  # flaml log file
    "seed": 22,
    "verbose" : 0
      
           # random seed 22  786
}
automl.fit(X_train=X_train, y_train=y_train, **settings)
'''retrieve best config and best learner'''
print('Best ML leaner:', automl.best_estimator)
print('Best hyperparmeter config:', automl.best_config)
print('Best accuracy on validation data: {0:.4g}'.format(1-automl.best_loss))
print('Training duration of best run: {0:.4g} s'.format(automl.best_config_train_time))

Everything works fine up to the above point. Now when I predict on test data using as

y_pred = automl.predict(test_sprk)
print('Predicted labels', y_pred)