microsoft/SynapseML

Featurizer should provide option to pass through missing values as Double.NaN instead of removing rows (currently the default)

Open

#304 创建于 2018年5月18日

在 GitHub 查看
 (5 评论) (0 反应) (1 负责人)Scala (5,228 star) (861 fork)batch import
enhancementgood first issuehelp wanted

描述

Hi! Using lightGBM I faced another problem. I'm not sure if it is bug or feature :) but in our data we have a lot of empty values, so before we used sparse vector to store features, and it worked fine with our previous lib. But when i tried to use featurizer, that you provide - i mentioned, that you skip all raws if any nulls are presents as a feature. you can see it in example in attachment. So is it possible to have sparse feature vector for lightGBM training?

https://gist.github.com/ekaterina-sereda-rf/929183b9bcbbf5baf15eec3e81329992

贡献者指南

技术栈
scala
领域
machine learningdata
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
basic ScalaSpark ML concepts
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
40
研究方向
Examine the Featurizer implementation in the SynapseML codebase, likely under src/main/scala. The current behavior drops rows with null features; the issue requests an option to retain missing values as Double.NaN. Look for the Featurizer class or object, and identify where the null handling logic is. The change involves adding a configuration parameter (e.g., setHandleMissing) and modifying the transform logic to either drop rows or pass NaN. Since the issue has an assignee and is old, it's advisable to check if any related PRs exist or if the maintainers have provided guidance in the comments.