Consider adding a trend categorisation component to evaluation
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
Research direction
No files, tests, or entry points are named. Start by locating the forecast evaluation code and how posterior draws and hospital-admission observations are represented; the work is done when trend categories are defined and evaluated with a Brier score.
Written by the indexing model from the issue text.
Description
We could classify each forecast from each model as increasing, decreasing, or stable based on the trajectory of hospital admissions.
This could either be something like probability increasing, probability decreasing, probability stable (which would be based on the posterior draws)
Then we could evaluate the trend by categorising the observation into one of 3 bins and using the Brier score to evaluate.
- Dominant language
- R
- Stars
- 4
- Forks
- 1
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 7
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from epiforecasts/evalwwforecasts
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
epiforecasts/evalwwforecasts#127 ·
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
epiforecasts/evalwwforecasts#123 ·
-
future work
Difficulty 5/5 Over a week Newbie friendliness 35/100
epiforecasts/evalwwforecasts#118 · 5 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
epiforecasts/evalwwforecasts#116 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 45/100
epiforecasts/evalwwforecasts#115 ·
All issues in epiforecasts/evalwwforecasts
Similar issues
-
dbRemoveTable() fails on SQLite with 'near "TRUNCATE": syntax error' and leaves the table in placeOpen
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
OHDSI/DatabaseConnector#348 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 71/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100