combust/mleap

Integrate with spark-nlp

Open

#292 opened on 2017年10月21日

GitHub で見る
 (1 comment) (0 reactions) (0 assignees)Scala (1,461 stars) (312 forks)batch import
enhancementepichelp wanted

説明

spark-nlp

Is a library that was developed by john snow labs to offer common transformers for NLP to Spark. It would be incredibly useful to be able to train these models and then deploy them with MLeap.

https://github.com/JohnSnowLabs/spark-nlp

We still need to outline the work in terms of which transformers and models we need to offer support for and if we will include them in core modules or create separate modules for NLP.

コントリビューターガイド

技術スタック
scalaspark
領域
datamachine learning
Issue 種別
feature
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
5
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
over 1 week
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
needs investigation
前提条件
ScalaApache SparkNLP basicsMLeap internals
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
15
調査方針
Investigate the current MLeap module structure in the combust/mleap repository, particularly how existing transformers are defined and serialized. Review the spark nlp library by John Snow Labs (github.com/JohnSnowLabs/spark nlp) to identify which transformers and models are most commonly used. Then propose a plan to either extend core MLeap modules or create separate NLP specific modules, referencing specific files such as MleapTransformer.scala or similar. The discussion in the issue indicates that the scope needs to be defined first.