Cannot exclude members to pickle in metaclass
还没有人认领这个 Issue。
评估
调研方向
首先在提供的 SingletonMeta 示例中重现该故障,重点关注 test_singleton_pickle 和最后的 cloudpickle.dumps(test_singleton) 调用。阅读 cloudpickle 如何处理元类和 getstate 方法,然后验证实例可以在不尝试 pickle 元类的 RLock 的情况下进行转储和加载,同时保留所示的 assertions。
由索引模型根据 Issue 内容生成。
描述
In the example below, I implemented __getstate__(cls) that returns empty dict
in order not to pickle cls.INSTANCES and cls.LOCK members of metaclass SingletonMeta.
But it seems that cloudpickle still tries to pickle those members
and fails to pickle since threading.RLock is not picklable.
import cloudpickle
from threading import RLock
class SingletonMeta(type):
def __new__(cls, name, bases, namespace):
return super().__new__(cls, name, bases, namespace)
def __init__(cls, name, bases, namespace):
super().__init__(name, bases, namespace)
cls.INSTANCES = {}
cls.LOCK = RLock()
def __call__(cls, *args, **kwargs):
key = cls.__getkey__(*args, **kwargs)
key = tuple(key.items())
if key not in cls.INSTANCES:
with cls.LOCK:
if key not in cls.INSTANCES:
cls.INSTANCES[key] = super().__call__(*args, **kwargs)
instance = cls.INSTANCES[key]
return instance
def __getstate__(cls):
# don't want to pickle cls.INSTANCES and cls.LOCK
return {}
def __setstate__(cls, state):
pass
class SingletonClass(metaclass=SingletonMeta):
def __init__(self, key):
self.key = key
self.transient = [self.key]
def __getstate__(self):
state = self.__dict__.copy()
del state["transient"]
return state
def __setstate__(self, state):
for name, value in state.items():
setattr(self, name, value)
self.transient = [self.key]
@classmethod
def __getkey__(cls, key):
return {"key": key}
# singleton implementation works fine as long as metaclass = SingletonMeta
def test_singleton():
singleton_instance1 = SingletonClass(0)
singleton_instance2 = SingletonClass(0)
singleton_instance3 = SingletonClass(1)
assert singleton_instance1 is singleton_instance2
assert singleton_instance1 is not singleton_instance3
# pickling instances works fine if metaclass != SingletonMeta
# but raises `TypeError: cannot pickle '_thread.RLock' object` if metaclass = SingletonMeta
def test_singleton_pickle():
singleton_instance1 = SingletonClass(0)
singleton_instance1_loaded = cloudpickle.loads(
cloudpickle.dumps(singleton_instance1)
)
assert singleton_instance1 is not singleton_instance1_loaded
assert singleton_instance1.key == singleton_instance1_loaded.key
assert singleton_instance1.transient == singleton_instance1_loaded.transient
if __name__ == "__main__":
# singleton implementation works fine as long as metaclass = SingletonMeta
test_singleton()
# pickling instances works fine if metaclass != SingletonMeta
# but raises `TypeError: cannot pickle '_thread.RLock' object` if metaclass = SingletonMeta
test_singleton_pickle()
# raises "TypeError: cannot pickle '_thread.RLock' object"
cloudpickle.dumps(test_singleton)
- 主要语言
- Python
- 星标
- 1.9k
- 派生
- 197
- 平均合并
- 1 天 10 小时
- 30 天内合并 PR
- 1
环境准备
我们还没有检查这个项目的环境配置文件。先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
cloudpipe/cloudpickle 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
cloudpipe/cloudpickle#593 · 1 条评论 ·
-
难度 4/5 3-5 天 新手友好度 42/100
cloudpipe/cloudpickle#595 ·
-
难度 3/5 1-2 天 新手友好度 65/100
cloudpipe/cloudpickle#592 · 2 条评论 ·
-
难度 5/5 一周以上 新手友好度 35/100
cloudpipe/cloudpickle#589 ·
-
难度 5/5 一周以上 新手友好度 20/100
cloudpipe/cloudpickle#587 ·
查看 cloudpipe/cloudpickle 的全部 Issue
相似的 Issue
-
good first issue
难度 2/5 1-3 小时 新手友好度 88/100
-
难度 2/5 1-3 小时 新手友好度 88/100
vllm-project/vllm-metal#822 ·
维护者通常 1 天内回复
-
vector-store
难度 1/5 1-3 小时 新手友好度 90/100
维护者通常 1 天内回复
-
[Bug]: chunk_span_bounds and _validated_chunk_spans reject Pydantic models ChunkSpan and AudioFile未关闭
难度 2/5 1-3 小时 新手友好度 78/100
BasedHardware/omi#19047 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复