Move the runner into a separate package with minimal dependencies · bloomberg/memray#557

(10 评论) (0 反应) (0 负责人)Python (11,796 star) (383 fork)batch import

Packagingenhancementgood first issue

描述

Is there an existing proposal for this?

I have searched the existing proposals

Is your feature request related to a problem?

The system I am currently working on involves some confidential customer data to which regular developers do not have access. Due to the complex nature of ML algorithms and libraries involved, sometimes we encounter anomalous memory usage which we cannot easily reproduce with our test data set. I am considering integration of some memory profiling mechanism, possibly involving memray which would be selectively enabled in production environment to help us to get insights into problematic cases without requiring access to customer data.

For multiple reasons (security, reliability, docker image size to name a few) we try to minimize the number of our dependencies. Memray has a number of dependencies which are unnecessary for trace collection:

https://github.com/bloomberg/memray/blob/578e02d20bc63bddc0fc25c28bf7569519b89838/setup.py#L91-L96

and those have their own dependencies as well.

Describe the solution you'd like

It would be great to split memray into two packages, tentatively named memray-core and memray. The former would only be able to perform collection, and the later would do everything memray does today.

Alternatives you considered

Another possibility would be having all the dependencies not related to memray.Tracker to be under "extras" but that would probably be a nuisance for most ordinary users who would want a simple pip install memray to install everything.

贡献者指南

技术栈: python
领域: build system
议题类型: feature
难度: 3
预计时间: 1-2 days
活动状态: stale
清晰度: clear
前置要求: Python packaging knowledgefamiliarity with setuptools
新手友好度: 35
研究方向: The issue requests splitting memray into two packages: memray core (minimal dependencies for trace collection) and memray (full features). The current setup.py at https://github.com/bloomberg/memray/blob/578e02d20bc63bddc0fc25c28bf7569519b89838/setup.py contains all dependencies. Research should focus on identifying which dependencies are essential for the core tracker and how to restructure the project to support two packages. Look at existing Python package splitting patterns like those used by Werkzeug or SQLAlchemy, and consider backward compatibility and release management.