sql: LIKE '%_' fails with "invalid encoding of the last character" in the row-by-row engine but matches in the vectorized engine when the string ends in a valid U+FFFD
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 69/100
Research direction
Read pkg/sql/sem/eval/match.go, especially optimizedLikeFunc near line 360, and compare its UTF-8 checks with the branches near lines 278, 339, and 381. Start by checking how utf8.DecodeLastRuneInString distinguishes a valid U+FFFD from malformed bytes; reproduce with the SQL in the issue using vectorize = off and on. Done means the row-by-row engine matches the vectorized result for the reported patterns and malformed input still gets the appropriate behavior.
Written by the indexing model from the issue text.
Description
Describe the problem
When a STRING value contains the valid character U+FFFD (the Unicode replacement character, encoded as EF BF BD) as its last character, SELECT s FROM t WHERE s LIKE '%_' fails in the row-by-row engine with XXUUU: invalid encoding of the last character in string, while the vectorized engine returns the row. The same happens for other patterns that start or end with the _ wildcard, for example LIKE '_', LIKE '_x%' and NOT ILIKE '_a'. _ matches any single character and a valid U+FFFD is a single character, so '%_' should match and the row should be returned.
To Reproduce
Deployment: a single-node CockroachDB cluster started with cockroach start-single-node --insecure (tested: v26.2.7, 1 node, default settings except sql.stats.automatic_collection.enabled = false). Run the script in an empty database. The script runs the query twice in one session: first with vectorize = off (row-by-row engine), then with vectorize = on (vectorized engine).
The inserted value is the single character U+FFFD (valid UTF-8 bytes EF BF BD).
SET TIME ZONE 'UTC';
CREATE TABLE t (s STRING PRIMARY KEY);
INSERT INTO t VALUES ('�');
-- row-by-row engine
SET vectorize = off;
EXPLAIN (VERBOSE) SELECT s FROM t WHERE s LIKE '%_';
SELECT s FROM t WHERE s LIKE '%_';
-- vectorized engine
SET vectorize = on;
EXPLAIN (VERBOSE) SELECT s FROM t WHERE s LIKE '%_';
SELECT s FROM t WHERE s LIKE '%_';
Expected behavior
The row is returned, as the vectorized engine does. The CockroachDB v26.2 documentation (String pattern matching, scalar-expressions) says: "Evaluates both expressions as strings, then tests whether the string on the left matches the pattern given on the right. Returns TRUE if a match is found or FALSE otherwise ... Patterns can contain _ to match any single character, or % to match any sequence of zero or more characters." The PostgreSQL 18.6 manual (pattern matching with LIKE) says the same: "An underscore (_) in pattern stands for (matches) any single character". A valid U+FFFD stored in the string is one character, so _ must match it. PostgreSQL 18.6 also returns the row for this query.
PostgreSQL 18.6 (Debian 18.6-1.pgdg12+2):
SELECT s FROM t WHERE s LIKE '%_';
psql:<stdin>:5: ERROR: 42P01: relation "t" does not exist
LINE 1: SELECT s FROM t WHERE s LIKE '%_';
^
LOCATION: parserOpenTable, parse_relation.c:1498
Additional data / screenshots
With vectorize = off the query fails with SQLSTATE XXUUU: invalid encoding of the last character in string �. With vectorize = on it returns one row containing �.
-- row-by-row engine
EXPLAIN (VERBOSE) SELECT s FROM t WHERE s LIKE '%_';
info
--------------------------------------------------
distribution: local
vectorized: false
• filter
│ columns: (s)
│ estimated row count: 333 (missing stats)
│ filter: s LIKE '%_'
│
└── • scan
columns: (s)
estimated row count: 1,000 (missing stats)
table: t@t_pkey
spans: FULL SCAN
(13 rows)
SELECT s FROM t WHERE s LIKE '%_';
psql:<stdin>:8: ERROR: XXUUU: invalid encoding of the last character in string �
LOCATION: func7, match.go:361
-- vectorized engine
EXPLAIN (VERBOSE) SELECT s FROM t WHERE s LIKE '%_';
info
--------------------------------------------------
distribution: local
vectorized: true
• filter
│ columns: (s)
│ estimated row count: 333 (missing stats)
│ filter: s LIKE '%_'
│
└── • scan
columns: (s)
estimated row count: 1,000 (missing stats)
table: t@t_pkey
spans: FULL SCAN
(13 rows)
SELECT s FROM t WHERE s LIKE '%_';
s
---
�
(1 row)
Environment:
- CockroachDB version:
CockroachDB CCL v26.2.7-dev-8652ac22d9f90501e48ae4f24efd7faab33cf7c5 (x86_64-pc-linux-gnu, built 2026/10/23 21:35:41, go1.25.13)
- Server OS: Linux 6.8.0-138-generic x86_64, AMD EPYC 7J13 64-Core Processor
- Client app:
psql
Additional context
In the row-by-row engine, optimizedLikeFunc builds a fast-path closure for patterns that end in _ (such as %_). The closure calls utf8.DecodeLastRuneInString and rejects the string when the decoded rune equals utf8.RuneError, but it discards the returned width. A valid U+FFFD (width 3) and a genuinely malformed last byte (RuneError, width 1) therefore look the same, and a valid U+FFFD is reported as an invalid encoding. The other branches of the same function use the same pattern (for example around lines 278, 339 and 381). When optimizedLikeFunc returns nil, the row-by-row engine matches the same pattern with the regular expression built by ConvertLikeToRegexp, which treats U+FFFD as an ordinary character. The vectorized engine does not use the byte fast path for such patterns: GetLikeOperatorType falls back to LikeRegexp when the pattern starts or ends with the _ wildcard, so it is not affected.
- CockroachDB: pkg/sql/sem/eval/match.go#L360 —
optimizedLikeFunc - CockroachDB: pkg/sql/colexec/colexeccmp/like_ops.go#L101 —
GetLikeOperatorType
Found with AI assistance.
Jira issue: CRDB-70793
- Dominant language
- Go
- Stars
- 32.6k
- Forks
- 4.2k
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cockroachdb/cockroach
-
C-bug O-community X-blathers-untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
cockroachdb/cockroach#176245 · 1 comment ·
-
sql: pg_database.dattablespace is 0 but pg_tablespace.oid is 1663, so they no longer joinPossibly taken @1AbdullahButt claimed this 14 days ago. OpenC-bug O-community X-blathers-triaged X-move-internal
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
cockroachdb/cockroach#176228 · 1 comment ·
-
PG compat: overflow in integer power calculation triggers incompatible SQLSTATEMay be free again @darshpatel1052 claimed this 18 days ago, and no pull request is open. OpenC-bug O-community X-blathers-untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cockroachdb/cockroach#176223 · 2 comments ·
-
util/log: bufferedSinkCloser.Close leaks a goroutine after timeoutPossibly taken @dajiaohuang claimed this 14 days ago. OpenC-bug O-community X-blathers-untriaged
Difficulty 1/5 Under an hour Newbie friendliness 84/100
cockroachdb/cockroach#176210 · 1 comment ·
-
C-bug O-community T-sql-queries X-blathers-triaged X-move-internal
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
cockroachdb/cockroach#176150 · 1 comment ·
All issues in cockroachdb/cockroach
Similar issues
-
area/import-export area/projects concern/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
go-vikunja/vikunja#4455 · 1 comment ·
Maintainers usually reply within 2 days
-
Dependabot PRs no longer merge automaticallyPossibly taken @dwisiswant0 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/urlfinder#160 ·
Maintainers usually reply within 1 day
-
database
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
chnm/legal-modernism#450 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
bug ready-for-triage ready-to-code
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
RedHatInsights/insights-ingress-go#946 · 4 comments ·
Maintainers usually reply within 1 day