Allow non-numbers as entity ids
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start by reviewing the entid function and the partially implemented IDB, ISearch, and IIndexAccess protocols in the MapToDatascript adapter described in the issue. Check the linked pull request and prior issue discussion before deciding whether flexible entity IDs fit the data model; done requires resolving the design and testing all ramifications across supported data sources.
Written by the indexing model from the issue text.
Description
This has come up in a couple of issues before, both by developers being against it and for it. To wit: https://github.com/tonsky/datascript/pull/8#issue-32457706 https://github.com/tonsky/datascript/issues/56#issuecomment-73641241
Here's my case for a dynamic entity id type, rather than enforcing number?, as is done in the entid function.
I tried to integrate the high churn real-time data stream that my process receives and updates a in-memory map with. This map has something like the following, pretty common structure:
{ "key1"
{ "attrA" "foo", "attrB" "bar", ... }
"key2"
{ "attrB" "foo", "attrX" "bar", ... }
}
It was easy enough to hook this map up to Datascript (Spossiba Tonsky, you ЯOCK! Datomic in contrast doesn't have published protocols...) by creating a MapToDatascript adapter, just implementing (partially, for the time being) IDB, ISearch, and IIndexAccess, and then query it like so, joining it with a real datascript DB:
(ds/q '[
:find (pull $ ?e [*]) (pull $m ?attr-x [*])
:in $ $m
:where [$ ?e :attr-x ?attr-x]]
@ds-db my-map-adapter)
Except, I had to tweak datascript's entid function - in fact simplifying it - in order to allow string eids, or just any type, for that matter.
Now I can of course create some number out of thin air and attach it to the entity to serve as the entity id, but that is another level of mapping that has to go on on every change of my data, which is thousands per second. What's worse, I now have another id that I need to keep consistent in the face of a machine going down and some failover machine taking over. That complicates things quite a bit.
So why not keep entity ids as flexible - for non-datascript data sources at least - as the datalog queries themselves, that don't care if your patterns are shaped [?e ?a ?v] or something completely different.
That way, integrating datascripts datalog engine with arbitrary data sources will be a lot easier, and depending on the case, more performant.
Is this a route you'd like to go in Datascript? Or is the datascript data model you want other sources to adhere to? In the former case, I'd have a more closed look at the sources and wrap this in a proper pull request, after checking and testing all the ramifications.
- Dominant language
- Clojure
- Stars
- 5.8k
- Forks
- 318
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from tonsky/datascript
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
tonsky/datascript#498 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
tonsky/datascript#489 ·
-
Stack overflow when transacting :db.type/tupleAttrs with a :db.type/ref attr through :db.fn/call Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
tonsky/datascript#483 · 2 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
tonsky/datascript#470 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
tonsky/datascript#441 · 1 comment · 3 reactions ·
All issues in tonsky/datascript
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
.Needs Triage .Team/UXWest Priority:P3 Type:New Feature
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
needs triage
Difficulty 1/5 Under an hour Newbie friendliness 90/100