Non-indexed JSONb operations
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
Research direction
Start in lib/ecto/adapters/postgres/connection.ex around line 1054 and reproduce the two Repo.to_sql examples comparing literal and pinned JSONB values. Review how PostgreSQL JSONB paths, pinned values, equality, and containment are represented; done means the proposed non-literal cases generate GIN-index-compatible SQL without regressing existing literal queries.
Written by the indexing model from the issue text.
Description
Elixir version
Erlang/OTP 27 [erts-15.2.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit] Elixir 1.18.3 (compiled with Erlang/OTP 27)
Database and Version
PostgreSQL 17.4 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 14.2.1 20240912 (Red Hat 14.2.1-3), 64-bit
Ecto Versions
3.12.5
Database Adapter and Versions (postgrex, myxql, etc)
0.20.0
Current behavior
When comparing jsonb paths with literals in where, Ecto generates efficient query using @>:
iex(1)> q = from j in Oban.Job, where: j.args["event_id"] == "123"
iex(2)> Repo.to_sql(:all, q) |> elem(0) |> IO.puts()
SELECT ... FROM "oban_jobs" AS o0 WHERE ((o0."args"@>'{"event_id": "123"}'))
However, if doing the same, but interpolating the value, e.g.:
iex(1)> q = from j in Oban.Job, where: j.args["event_id"] == ^"123"
iex(2)> Repo.to_sql(:all, q) |> elem(0) |> IO.puts()
SELECT ... FROM "oban_jobs" AS o0 WHERE ((o0."args"#>'{"event_id"}') = $1)
Ecto generates the query using #> operator, which isn't supported by GIN indexes
Expected behavior
It seems that this inefficiency comes from https://github.com/elixir-ecto/ecto_sql/blob/ad5e31c13b034564ff8bf3783bfb2e8ab7c0e6d0/lib/ecto/adapters/postgres/connection.ex#L1054, which is restricted only to literals, and not interpolations.
My proposal is to add support for pinned vars and other db columns with following cases:
-
data known in elixir:
SELECT ... FROM "oban_jobs" AS o0 WHERE (o0."args" @@ '$.event_id #{operator} #{Jason.encode value}') -
equality comparison:
SELECT ... FROM "oban_jobs" AS o0 WHERE (o0."args" @> jsonb_build_object($1, jsonb_build_object($2, ...)))where $1..$n-1 is path items and $n is the value.
Maybe it would be more optimal case for elixir-data equality comparisons since the query can be prepared -
containment check
iex(1)> q = from j in Oban.Job, where: %{event_id: "123"} in j.args iex(2)> Repo.to_sql(:all, q) |> elem(0) |> IO.puts() SELECT ... FROM "oban_jobs" AS o0 WHERE (o0."args" @> jsonb_build_object('event_id', '123'))The idea is to re-build the object using
jsonb_build_objectandjsonb_build_arraywhich would allow for specific values to be non-literals
- Dominant language
- Elixir
- Stars
- 657
- Forks
- 345
- Avg merge
- 1h 59m
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from elixir-ecto/ecto_sql
-
Kind:Enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
elixir-ecto/ecto_sql#637 · 3 reactions ·
-
Kind:Bug
Difficulty 4/5 3-5 days Newbie friendliness 35/100
elixir-ecto/ecto_sql#602 · 4 comments ·
All issues in elixir-ecto/ecto_sql
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
phoenixframework/phoenix#6847 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
moderation
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
beyond-all-reason/teiserver#1584 · 1 comment ·