feathr-ai/feathr

API to query features materialized to offline

Open

Aperta il 20 mag 2022

Vedi su GitHub
 (3 commenti) (2 reazioni) (0 assegnatari)Scala (1929 star) (244 fork)batch import
good first issue

Descrizione

Today for features materialized to online(Redis), we provide a simple API to query it.

Some users may want to materialize features to offline store(delta lake) and query it later on. This API doesn't exist yet.

We should add this API so users can query features generated by Feathr.

The API should be consistent with online API:

    res = client.get_online_features(online_test_table, '265', [
                                     'f_location_avg_fare', 'f_location_max_fare'])

    res = client.multi_get_online_features(online_test_table,
                                           ['239', '265'],
                                           ['f_location_avg_fare', 'f_location_max_fare'])

The API can be like(this is still up for discussion):

feathrClient.getFeatures([f1, f2], feature_table)

feathrClient.joinFeature([f1, f2], feature_table, observation_table)

Guida contributor

Tech stack
pythonscala
Dominio
apidatabackend
Tipo issue
feature
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
4
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
3-5 days
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
mostly clear
Prerequisiti
Feathr online API knowledgeDelta Lake understandingPython client development
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
30
Direzione di ricerca
The issue proposes adding two new APIs to the Python client: getFeatures and joinFeature. The implementation should follow the pattern of the existing online API (get online features, multi get online features). Look at the Feathr client source code, particularly the online query implementation, to understand the required remote calls. Since the offline store is Delta Lake, incorporate Spark or Delta Lake reading logic. Review the discussion in the issue comments for additional requirements.