feast-dev/feast

Latest Only option for Historical Retrieval

Open

#1,687 opened on Jul 5, 2021

View on GitHub
 (12 comments) (1 reaction) (0 assignees)Python (5,029 stars) (896 forks)batch import
Community Contribution Neededgood first issuekeep-openkind/featurepriority/p1

Description

Is your feature request related to a problem? Please describe.

In many batch workflows, it is worthwhile to retrieve the latest features by entity only. This is useful from the purposes of both production and backtesting purposes.

E.g. if I have an hourly/daily batch which goes through our whole customer base to find fraudulent customers, we wouldn't really use the online store for this.

Describe the solution you'd like

Allow users to specify an entity set extracted from a feature view should have an option to be deduplicated by latest. Depends on #1611

my_daily_batch_scoring_df = store.get_latest_features(
    entity_df = "my_df", 
    feature_refs = [...],
)

Additional context Linked issue #1611

Contributor guide

Tech stack
python
Domain
datamachine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Familiarity with Feast feature storeUnderstanding of historical retrieval APIKnowledge of entity sets and deduplication
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
The feature request proposes a new method `get latest features` to deduplicate historical features by entity. Implementation depends on resolving issue #1611 first. The existing `get historical features` method in the Python SDK (likely in `sdk/python/feast/inference.py` or similar) should be studied. Comments on the issue may contain discussion on API design. Focus on understanding how entity sets and feature views are currently retrieved and how deduplication by latest timestamp would fit.