combust/mleap

Add support for spark-extensions transformers in Pyspark

Open

#570 建立於 2019年9月24日

在 GitHub 查看
 (10 留言) (0 反應) (0 負責人)Scala (1,461 star) (312 fork)batch import
enhancementepichelp wanted

描述

Creating this parent issue to track progress on adding support for spark-extensions transformers in Pyspark.

  • Support for StringMap
  • Support for MathBinary
  • Support for MathUnary
  • Support for WordLengthFilter
  • Support for MultinomialLabeler
  • Support for Imputer
  • Support for OneVsRest
  • Support for SupportVectorMachine
  • Add unit test support for Pyspark transformers

Related issues:

There are currently two examples of custom transformers in Pyspark under https://github.com/combust/mleap/pull/568.

貢獻者指南

技術棧
python
領域
machine learningdata
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
half day
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
stale
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
mostly clear
前置要求
Basic PySparkMLeap familiarity
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
50
研究方向
Examine the existing custom transformers implemented in PR #568 (StringMap, MathBinary, MathUnary) as examples. Then implement one of the remaining transformers (WordLengthFilter, MultinomialLabeler, Imputer, OneVsRest, SupportVectorMachine) following the same pattern. Refer to related issues #551, #552, #477, #495 for additional context and requirements. Ensure that unit tests are written for the new transformer.