Update `Recipe` contract to allow for more generic transformations
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Domain
- machine-learning
Research direction
Start by reading the Recipe, processor lifecycle, and RecipeExecution APIs, then trace how model forward, fit, and predict calls are wired. The issue identifies no files or tests, so locate existing processor and recipe tests before proposing an acceptance test for forecasts at different query positions and clear failure when required coordinates are absent. The coordinate shapes, compatibility contract, and related-table behavior still need design decisions.
Written by the indexing model from the issue text.
Description
My initial goal was to extend Recipe such that a transformation such as conditional detrending (but not only) could fit there. I'd like to find a way to make it as general and clean as possible.
Proposal for discussion
Recipe can fit a transformation on context rows and invert it on model predictions, but processor calls cannot currently receive the position of each row. A fitted trend therefore cannot be evaluated reliably at future rows, especially when positions have gaps or forecasts are requested out of order.
Here is an illustrative call site. ConditionalLinearDetrend does not exist yet; the goal is a contract that would also support other transformations needing row coordinates.
import torch
import sdm
context_positions = torch.tensor([0., 1., 3., 4.])
query_positions = torch.tensor([6., 9.])
targets = sdm.TableTensor.from_tensor(
torch.tensor([
[10., 4.],
[12., 7.],
[16., 4.],
[18., 7.],
])
columns=["load", "alternating"],
)
recipe = sdm.Recipe(
target=ConditionalLinearDetrend(residual_std_ratio=0.5)
)
recipe.target.fit_transform(
targets, row_positions=context_positions
)
raw_forecast = sdm.TableTensor.from_tensor(
torch.tensor([[0., 5.], [0., 6.]]),
columns=["load", "alternating"],
)
forecast = recipe.target.inverse_transform(
raw_forecast, row_positions=query_positions
)
assert forecast.numerical.tolist() == [[22., 5.], [28., 6.]]
The load series follows 10 + 2 × position and meets the detrending condition. The alternating series does not, so its predictions remain unchanged. Fitting uses context values only; inversion evaluates the fitted trend at the supplied query positions.
Proposed direction
- Accept optional row coordinates in the processor lifecycle and pass them through
RecipeExecutionand modelforward,fit, andpredictcalls. - Let processors that need coordinates require them explicitly. Existing processors and recipes should continue to work without them.
Questions to settle
- What shapes and broadcasting rules should coordinates support for batched tables and ensemble members? Should they represent ordinal steps, arbitrary numeric coordinates, or both?
- How should a processor declare that coordinates are required while preserving existing processor subclasses?
- Related tables have their own row axes. How should their coordinates be supplied?
- Forecasts with several quantiles per target also need an explicit target-to-output mapping for inversion. Should that be a separate issue?
A useful acceptance test would fit once on context, then request forecasts at different query positions in either order. Each result should depend on its supplied positions, while a processor called without required coordinates should fail clearly.
- Dominant language
- Python
- Stars
- 316
- Forks
- 27
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 157
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/structured-data-models
-
Kumo-Tabular weights unreachable behind restrictive egress proxies (Xet + plain HTTP both fail)Possibly taken @JGagacheva claimed this 2 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
NVIDIA/structured-data-models#1091 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
NVIDIA/structured-data-models#1030 · 6 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
NVIDIA/structured-data-models#1022 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
NVIDIA/structured-data-models#947 · 1 reaction ·
Maintainers usually reply within 1 day
-
Add more ProcessorsOpen
Difficulty 4/5 3-5 days Newbie friendliness 35/100
NVIDIA/structured-data-models#336 · 1 comment ·
Maintainers usually reply within 1 day
All issues in NVIDIA/structured-data-models
Similar issues
-
customer-reported question
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Azure/azure-sdk-for-python#49548 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
btclib-org/btclib-wallet#409 ·
Maintainers usually reply within 1 day
-
Phase 2 PSA Capability & Career Profile
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
bcgov/digital-journeys#2310 ·
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
fastmachinelearning/hls4ml#1582 ·
Maintainers usually reply within 3 days
-
Sort modelsOpenenhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day