facebook/prophet

Decouple performance measures

Open

#2234 aperta il 22 lug 2022

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Python (4451 fork)batch import
enhancementgood first issue

Metriche repository

Star
 (17.686 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

Currently, diagnostics.performance_metrics() contains a function scoped list of available metrics: https://github.com/facebook/prophet/blob/95b1741d447c7283363475a63f1ceceaec2981c9/python/prophet/diagnostics.py#L365

This makes it impossible for a user to create their own performance measure calculator for inclusion without modifying the package code (monkey patching).

I propose a decorator-based, performance-measure registration system to decouple the list of potential performance measures from diagnostics.performance_metrics(). This would improve maintainability of the Prophet code in the case new performance measures are included within the package, and would allow users to register their own performance measures without monkey patching the package.

For example:

Existing:

def mse(df, w):
    se = (df['y'] - df['yhat']) ** 2
    if w < 0:
        return pd.DataFrame({'horizon': df['horizon'], 'mse': se})
    return rolling_mean_by_h(
        x=se.values, h=df['horizon'].values, w=w, name='mse'
    ) 

Would become

@performance_metric
def mse(df, w):
    se = (df['y'] - df['yhat']) ** 2
    if w < 0:
        return pd.DataFrame({'horizon': df['horizon'], 'mse': se})
    return rolling_mean_by_h(
        x=se.values, h=df['horizon'].values, w=w, name='mse'
    ) 

With:

PERFORMANCE_METRICS=dict()
def performance_metric(func):
    PERFORMANCE_METRICS[func.__name__] = func
    return func

and with changes to diagnostics.performance_metrics() to reference the new PERFORMANCE_METRICS dictionary.

Guida contributor