feathr-ai/feathr

[BUG] non-feature columns get converted into string columns when get result dataframe

Open

#942 opened on 2023年1月5日

GitHub で見る
 (0 comments) (0 reactions) (0 assignees)Scala (1,929 stars) (244 forks)batch import
buggood first issue

説明

Willingness to contribute

No. I cannot contribute a bug fix at this time.

Feathr version

0.9.0

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 20.0): Linux Ubuntu 20.0)
  • Python version: 3.8
  • Spark version, if reporting runtime issue: 3.3.1

Describe the problem

non-feature columns of the original source data get converted into string column, e.g. 0 (int) -> "0", True (bool) -> "True" string values when getting the feature result.

Tracking information

No response

Code to reproduce bug

No response

What component(s) does this bug affect?

  • Python Client: This is the client users use to interact with most of our API. Mostly written in Python.
  • Computation Engine: The computation engine that execute the actual feature join and generation work. Mostly in Scala and Spark.
  • Feature Registry API: The frontend API layer supports SQL, Purview(Atlas) as storage. The API layer is in Python(FAST API)
  • Feature Registry Web UI: The Web UI for feature registry. Written in React

コントリビューターガイド

技術スタック
pythonscala
領域
databackend
Issue 種別
bug
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
3
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
1-3 hours
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
mostly clear
前提条件
Scala basicsSpark concepts
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
30
調査方針
Investigate the Computation Engine component (Scala Spark) to identify where non feature columns are cast to strings. Look at the feature join and generation logic, possibly in Scala files under the computation engine directory. Check for explicit type conversions or schema inference issues. Since there is no reproduction code, you may need to create a minimal test case to replicate the bug.