apache/beam
Can't use side inputs with Combine.PerKeyWithHotKeyFanout
Offen
#20.637 geöffnet am 04.06.2022
P3bugcoregood first issuejava
Repository-Metriken
- Stars
- (7.313 Sterne)
- PR-Merge-Metriken
- (PR-Metriken ausstehend)
Beschreibung
PerKeyWithHotKeyFanout doesn't allow a user to specify a side input and access it in the CombineFn. When the function withHotKeyFanout is called, any sideinputs stored in the Combine.PerKey are dropped. (https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Combine.java#L1571):
public PerKeyWithHotKeyFanout<K, InputT, OutputT> withHotKeyFanout(
SerializableFunction<?
super K, Integer> hotKeyFanout) {
return new PerKeyWithHotKeyFanout<>(fn, fnDisplayData, hotKeyFanout);
}
We would like to be able to use side inputs and hot key fanout.
Imported from Jira BEAM-11005. Original Jira may contain additional context. Reported by: miagilepner.