combust/mleap

Add support for spark-extensions transformers in Pyspark

Open

#570 创建于 2019年9月24日

在 GitHub 查看
 (10 评论) (0 反应) (0 负责人)Scala (1,461 star) (312 fork)batch import
enhancementepichelp wanted

描述

Creating this parent issue to track progress on adding support for spark-extensions transformers in Pyspark.

  • Support for StringMap
  • Support for MathBinary
  • Support for MathUnary
  • Support for WordLengthFilter
  • Support for MultinomialLabeler
  • Support for Imputer
  • Support for OneVsRest
  • Support for SupportVectorMachine
  • Add unit test support for Pyspark transformers

Related issues:

There are currently two examples of custom transformers in Pyspark under https://github.com/combust/mleap/pull/568.

贡献者指南

技术栈
python
领域
machine learningdata
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
mostly clear
前置要求
Basic PySparkMLeap familiarity
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
50
研究方向
Examine the existing custom transformers implemented in PR #568 (StringMap, MathBinary, MathUnary) as examples. Then implement one of the remaining transformers (WordLengthFilter, MultinomialLabeler, Imputer, OneVsRest, SupportVectorMachine) following the same pattern. Refer to related issues #551, #552, #477, #495 for additional context and requirements. Ensure that unit tests are written for the new transformer.