[Feature Request] `Frequency` merge operations
@bvenn is already working on this.
Since Apr 26, 2023.
Assessment
This issue has not been assessed yet.
Description
Merge operations for Maps
Data can be sorted into bins of predefined width using the Frequency or EmpiricalDistribution module. If two datasets are binned and should be merged afterwards, several merging strategies are possible. A simple merge of freqA and freqB is straightforward with keys that are present in freqA and freqB are replaced with the values of freqB.
let a =
[("k1",1);("k2",3)]
|> Map.ofList
let b =
[("k2",1);("k3",4)]
|> Map.ofList
merge a b
results in the following combination with ("k2",3) from a being replaced by ("k2",1) from b:
val it: Map<float,int> = map [("k1", 1); ("k2", 1); ("k3", 4)]
Generic formulation of merge operations
I'm in the process of adding a generic function that gets an additional function that handles key duplicates. E.g.:
add a b
resulting in the combination of a and b with ("k2",3) from a being added to ("k2",1) from b:
val it: Map<float,int> = map [("k1", 1); ("k2", 4); ("k3", 4)]
While this is trivial, I'm not sure how to handle a subtraction. Should the result from subtract a b result in:
- a)
val it: Map<float,int> = map [("k1", 1); ("k2", 2); ("k3", 4)]- counts from
aare subtracted by the corresponding values frombif keys are present in both maps - here the values of
athat are not present inbare untouched
- counts from
- b)
val it: Map<float,int> = map [("k1", 1); ("k2", 2); ("k3", -4)]- counts from
aare subtracted by the values fromb, even for keys that are not present ina
- counts from
The latter option (b) makes no sense to me since frequency counts should not be negative, but I cannot think of applications in which the result of (a) makes any sense. Maybe the subtract function is not the best to start with because in this post they implemented (a) with addition and multiplication examples. Especially for the addition, a and b would give the correct result and I think it is intuitive to just apply the function to values of keys that are present in both maps.
@HarryMcCarney, do you know use cases that use subtract? Do you have any thoughts about this? I would suggest to add version (a) to Frequency as well as EmpiricalDistribution
Additional remark: When applied to continuous data bandwidths must be equal, to not merge counts from overlapping bins!
- Dominant language
- F#
- Stars
- 227
- Forks
- 58
- Avg merge
- 55m
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from fslaborg/FSharp.Stats
-
Documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
fslaborg/FSharp.Stats#385 ·
-
automation Documentation repo-assist
Difficulty 1/5 Under an hour Newbie friendliness 1/100
fslaborg/FSharp.Stats#382 ·
-
question
Difficulty 5/5 Over a week Newbie friendliness 25/100
fslaborg/FSharp.Stats#355 · 18 comments ·
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 55/100
fslaborg/FSharp.Stats#354 · 2 comments ·
-
enhancement needs investigation
Difficulty 4/5 3-5 days Newbie friendliness 45/100
fslaborg/FSharp.Stats#344 · 4 comments ·