bruin-data/bruin
View on GitHubCreate Bronze-to-Silver Pipeline Template for BigQuery
Open
#1,220 opened on Oct 16, 2025
good first issue
Repository metrics
- Stars
- (1,639 stars)
- PR merge metrics
- (PR metrics pending)
Description
Create a new Bruin template that demonstrates a bronze-to-silver data pipeline pattern using BigQuery as the destination. This template will use either frankfurter or chess as the data source (since they don't require credentials) and showcase a complete ingestr -> transformation workflow.
This template should be able to initialized with bruin init template-name
more about bruin init on : https://getbruin.com/docs/bruin/commands/init.html
Template Structure
Bronze Layer (Ingestr Asset):
- Raw data ingestion using ingestr from frankfurter or chess API
- No transformations, just data extraction to BigQuery
Silver Layer (SQL or Python Asset):
- Aggregates and cleans bronze data from BigQuery tables
- Demonstrates typical transformation patterns
Implementation
Create templates/bronze-silver-bigquery/ with:
.bruin.ymlwith BigQuery connectionREADME.mdwith setup instructionspipeline.yml- Bronze asset:
assets/bronze_raw_data.asset.yml(ingestr from frankfurter/chess) - Silver asset:
assets/silver_aggregated.sql(BigQuery SQL transformation)
Acceptance Criteria
- Template created in
templates/bronze-silver-bigquery/ - Bronze layer uses ingestr asset to extract data from frankfurter or chess source (no credentials needed)
- Silver layer aggregates bronze data with BigQuery SQL transformations
- Documented with clear setup and usage instructions in templates documentations
- Can be initialized with
bruin init bronze-silver-bigquery -
bruin validatepasses successfully - Assets have meaningful quality checks (column checks, custom checks)
-
bruin runexecutes the full pipeline successfully
Reference Templates
- Existing templates: https://getbruin.com/docs/bruin/getting-started/templates.html
- Chess template: https://getbruin.com/docs/bruin/getting-started/templates-docs/chess-README.html
- Frankfurter template: https://getbruin.com/docs/bruin/getting-started/templates-docs/frankfurter-README.html