feathr-ai/feathr

[BUG] useLatestFeatureData doesn't use the latest feature data, but use "today's" feature data (which might not exist)

Open

#898 创建于 2022年12月2日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Scala (1,929 star) (244 fork)batch import
buggood first issue

描述

Willingness to contribute

No. I cannot contribute a bug fix at this time.

Feathr version

0.9.0

System information

NA

Describe the problem

When using joinTimeSettings: { useLatestFeatureData: true }, Feathr uses LocalDateTime.now() to get the join time range. This results in None features if the feature's date is not "today".

Expected behavior is to use the max timestamp value from the feature data source so that it actually joins the "latest" feature data available.

Tracking information

No response

Code to reproduce bug

No response

What component(s) does this bug affect?

  • Python Client: This is the client users use to interact with most of our API. Mostly written in Python.
  • Computation Engine: The computation engine that execute the actual feature join and generation work. Mostly in Scala and Spark.
  • Feature Registry API: The frontend API layer supports SQL, Purview(Atlas) as storage. The API layer is in Python(FAST API)
  • Feature Registry Web UI: The Web UI for feature registry. Written in React

贡献者指南

技术栈
scala
领域
backenddata
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
fresh
清晰度议题是否清楚说明期望改动、验收标准和下一步。
mostly clear
前置要求
Scala basicsSpark conceptsFeathr join semantics
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
35
研究方向
The bug is in the computation engine (Scala/Spark). The issue states that `useLatestFeatureData: true` causes `LocalDateTime.now()` to be used instead of the max timestamp from the feature data. Look for the join logic that handles `useLatestFeatureData` in the Scala source code, likely in the feature join or observation generation components. Search for `useLatestFeatureData` in the repository to locate relevant files. The fix would involve replacing the current date call with a query to get the maximum timestamp from the feature source.