yzhao062/pyod

Cannot save AutoEncoder

Open

#256 opened on Dec 6, 2020

View on GitHub
 (6 comments) (1 reaction) (1 assignee)Python (7,762 stars) (1,308 forks)batch import
buggood first issuehelp wanted

Description

The official instructions say to use joblib for pickling PyOD models.

This fails for AutoEncoders, or any other TensorFlow-backed model as far as I can tell. The error is:

>>> dump(model, 'model.joblib')
...
TypeError: can't pickle _thread.RLock objects

Note that it's not sufficient to save the underlying Keras Sequential model, since I need the methods & variables of BaseDetector (like .decision_scores_ or .decision_function().

Contributor guide

Tech stack
python
Domain
machine learning
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
PythonTensorFlowjoblib
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Investigate the AutoEncoder class in pyod/models/auto encoder.py. The error occurs because TensorFlow models contain non picklable objects. Consider implementing a custom save method that saves the Keras model weights and the detector's decision scores separately, then restores them on load. Check the existing model persistence documentation and any previous discussions in issue #256 for suggested approaches.