De-duplication agent de-duplicates on null values

If I set the deduplication agent property to {{A}} and pipe this through it (via the manual event generator from #1239 ) :

{
  "payloads": [
    {
      "A": "1",
      "B": "2"
    },
    {
      "C": "3",
      "D": "3"
    },
    {
      "A": "1",
      "B": "2"
    }
  ]
}

... I correctly only get {"A":"1","B":"2"} (once) and {"C":"3","D":"3"} out, although the dedupe agent memory shows :

{
  "properties": [
    "1",
    ""
  ]
}

... the second (empty) entry seems a little odd.

On the second run of the same set of events, I get nothing out of the dedupe agent, and I expected {"C":"3","D":"3"}

Event {"C":"3","D":"3"} had no property={{A}} so I'm guessing that the unexpected empty string in the de-dupe agent memory is the null value for {{A}} in the {"C":"3","D":"3"} event. In standard SQL, null doesn't match to null because it isn't a real value (although its often implemented internally as one); it means "I don't know" and matching events where you don't know the value of a specific property with all the other events where you don't know the value of the same property doesn't have a useful meaning. It looks like the de-dupe agent is implementing null properties as empty strings, and matching these empty strings. At the moment, if I set property={{A}} in a de-dupe agent and push event {"C":"3"} through then the following event {"D":"4"} is de-duped and not emitted because both share the same null value for {{A}}), which seems illogical.

I think in Oracle SQL, "" (an empty string) is treated the same as a null, but in SQL Server it is not, and I'm not sure which approach is better.

Issue first noticed when testing #1239