streaming_table: INSERT is never re-projected to match schema.yml's declared column order
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 83/100
Research direction
Start at macros/materializations/models/streaming_table.sql:57 and compare the materialized_table change in PR #84, focusing on get_select_subquery and the separate -ddl and main/INSERT call blocks; helpers.sql contains the related contract-gated macro. Add functional coverage for a streaming_table whose model SQL order differs from schema.yml, and confirm CI shows the INSERT matching the declared column order.
Written by the indexing model from the issue text.
Description
Summary
Observed in code, in passing while working on MTs. Needs validation.
streaming_table's long-running INSERT INTO {{ target_relation }} {{ sql }} (macros/materializations/models/streaming_table.sql:57) uses the model's SQL as-authored, with no re-projection to match the column order declared in schema.yml's columns: block.
This is the same bug class just fixed for materialized_table in PR #84 (which was missing get_select_subquery's reprojection in its CREATE OR ALTER MATERIALIZED TABLE ... AS SELECT), except here it's unconditional (not contract-gated) and still unfixed. No existing functional test for streaming_table exercises mismatched column order, so this gap isn't caught by CI.
Proposed fix
Add {%- set sql = get_select_subquery(sql) -%} once, at the materialization's top level between the -ddl call block and the main/INSERT call block (it can't go inside either {% call %}...{%- endcall %} body, since those are separate Jinja scopes and a set inside one wouldn't be visible to the other). Unlike the materialized_table fix, this shouldn't reuse render_contract_columns_and_reproject_sql (helpers.sql) as-is, since that macro is gated on contract.enforced and streaming_table's column rendering is unconditional -- reusing it would silently skip reprojection for the common case of a model that doesn't set contract={'enforced': true}.
- Dominant language
- Python
- Stars
- 9
- Forks
- 3
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 15
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from confluentinc/dbt-confluent
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
confluentinc/dbt-confluent#150 ·
-
Internal GFI
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
confluentinc/dbt-confluent#148 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Internal GFI
Difficulty 4/5 3-5 days Newbie friendliness 35/100
confluentinc/dbt-confluent#153 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 65/100
confluentinc/dbt-confluent#152 ·
All issues in confluentinc/dbt-confluent
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100