Add simple, self-contained SQL/DB examples for Kotlin DataFrame
I maintainer di solito rispondono entro 1 giorno
@zaleslaw ci sta già lavorando.
Dal 4/8/2026.
Valutazione
Questa issue non è ancora stata valutata.
Descrizione
Motivation
Today the Database + Kotlin DataFrame (+ Kandy) integration is only demonstrated by the external
KotlinDataFrame-SQL-Examples project. Problems:
- its notebooks are mostly deprecated;
- the flagship example (IMDB) needs a >100 MB MariaDB dump downloaded from Google Drive — the barrier to
entry is so high that basically nobody actually runs it.
We want a set of small, self-contained examples inside the dataframe repository: no external dumps, no
mandatory Docker where avoidable, so that "clone → run → see a DataFrame from a database" just works.
What already exists (foundation to reuse)
- The
dataframe-jdbcmodule supports H2, SQLite, PostgreSQL, MySQL, MariaDB, MS SQL, DuckDB.
Entry points indataframe-jdbc/src/main/kotlin/.../io/readJdbc.kt:
readSqlTable,readSqlQuery,readResultSet,readAllSqlTables,
Connection.readDataFrame(...),DbConnectionConfig.readDataFrame(...),DataSource.readDataFrame(...). - "Postgres inside H2" is built in:
H2(mode = Mode.PostgreSql)(.../io/db/H2.kt,MODE=PostgreSQL) reads
with PostgreSQL logic. Reference tests:dataframe-jdbc/src/test/kotlin/.../io/h2/. - Testcontainers will be wired up soon:
testcontainersTest/localDbTesttasks indataframe-jdbc/build.gradle.kts,
image versions ingradle/libs.versions.toml(postgres18-alpine, mysql9.7, mariadb12.3.2). - Examples are standalone projects under
examples/projects/<name>/; all are automatically compiled during
:test(debug mode) viabuild-logic/.../dfbuild.buildExampleProjects.gradle.kts. Related existing ones:
exposed,hibernate(ORM↔DataFrame); CSV+Kandy inkotlin-dataframe-plugin-gradle-example. - Backend guide:
docs/StardustDocs/topics/guides/Guide-for-backend-SQL-developers.md.
Scope — new examples
Each example is a new folder under examples/projects/: a standalone Gradle project with its own
settings.gradle.kts, a tiny embedded/"fun" dataset, a README.md, and a runnable main(). DB drivers are
runtime dependencies. A companion Jupyter notebook may be added per example where it works cleanly, but the
Gradle project is the primary, CI-tested artifact.
-
sql-csv-join— join a CSV file with a database table.
readCsv(<resource>)+readSqlTable(...)from H2/SQLite →join+groupBy/aggregation.
Shows DataFrame as a bridge between data sources. Includes a Kandy chart of the result. -
sql-sqlite— read from a bundled.sqlitefile in resources. Zero setup.
Connection.readDataFrame("SELECT ...")/readSqlTable. Small themed dataset. -
sql-postgres— one database, two ways to run it:- zero-Docker:
H2(Mode.PostgreSql)withMODE=PostgreSQL, schema + data,readSqlQuery/readResultSet; - optional "real PostgreSQL" via Testcontainers (image already in the project) as a separate
main/README section.
Includes a Kandy chart.
- zero-Docker:
-
sql-ktor— a minimal Ktor server that reads a database into aDataFrameon
request and serves it over HTTP (JSON viadataframe-jsontoJson, and/or a small HTML page). Uses embedded H2
so it runs with no external setup.
Out of scope: a Spring Boot + full schema-migration example (too heavy — pulls in Spring, which the repo has
zero of today, plus a migration tool). Track it separately if wanted.
Datasets
Two viable sources: reuse datasets already bundled in the repo, or generate a tiny dataset in code.
Rule of thumb:
- Read-only "read from a real DB" demos → reuse a ready-made, recognizable dataset. The repo already ships
the Chinook SQLite DB atexamples/projects/exposed/src/main/resources/chinook.db(a digital-music store:
artists,albums,tracks,genres,invoices,customers— multiple joinable tables, well known,
~1 MB, no download). Best default. - Demos that must own their schema/seeding (Postgres types, Testcontainers) → generate a small themed dataset
in code via inlineCREATE TABLE+INSERT(idempotent, identical for H2-in-postgres-mode and a real
container, no binary blob to check in). Keep it small (~20–50 rows) and "fun". Pattern to copy:
createPostgresTestDataindataframe-jdbc/src/test/kotlin/.../io/postgresTestBase.kt.
Per example:
sql-sqlite→ reusechinook.dbas-is. Zero generation; run a query (e.g. top-selling genres, tracks
per album) viaConnection.readDataFrame("SELECT ..."). Strongest ready-made fit.sql-csv-join→ combine a real DB table (Chinookartists/albums) with a small hand-authored CSV
that joins on a key (e.g.artist,country,formedYear). Generate the tiny CSV side by hand so the join key and
result are obvious. Alternative fully-ready combo:movies.csv(already in themoviesexample) + a generated
in-memory ratings table.sql-postgres→ generate a small themed schema+rows in code (inline SQL). The same seeding runs against
H2(Mode.PostgreSql)and against the Testcontainers Postgres — no file dependency — and lets us show a couple
of Postgres-flavored types/queries deliberately.sql-ktor→ generate and seed an embedded H2 at startup (so the app is fully self-contained), or reuse
chinook.dbread-only. Serve the resultingDataFrameas JSON viadataframe-jsontoJson.
Notes: Chinook is a public, redistributable sample DB and is already committed, so no licensing/size concern.
Avoid the test-only safe_moz_places_sample.sqlite (Firefox history — not illustrative) except for smoke tests.
Implementation
- For each example create
examples/projects/<name>/following existing ones
(examples/projects/exposed/,.../kotlin-dataframe-plugin-gradle-example/): ownsettings.gradle.kts
(rootProject.name),build.gradle.kts,gradle/libs.versions.toml,src/main/kotlin, resources,README.md. - Mirror each example into
examples/projects/dev/<name>/(dev mirror targeting the master library version). - In
build-logic/.../dfbuild.buildExampleProjects.gradle.ktsadd the new libraries (ktor, postgres/sqlite
drivers, testcontainers; kandy already present) toversionsToSyncandexampleDependencyUpdatesso version
sync anddependencyUpdatesare aware of them. Add new versions to the rootgradle/libs.versions.toml. - Update
examples/README.md: a section for the SQL/DB examples with links; mark the old
KotlinDataFrame-SQL-Examplesas "advanced / large dataset". - (Optional) add companion notebooks under
examples/notebooks/<name>/where they run cleanly, and cross-link
fromdocs/.../Guide-for-backend-SQL-developers.md.
Acceptance criteria
./gradlew runBuildAllExampleFolders -Pkotlin.dataframe.debug=truecompiles all new examples (release + dev).- Each example runs locally with no external data download; the Testcontainers variant runs only when Docker is present.
- Every example has a
README.mdexplaining what it shows and how to run it. examples/README.mdis updated.
- Lingua principale
- Kotlin
- Stelle
- 1.1k
- Fork
- 87
- Merge medio
- 2g 21h
- PR unite (30g)
- 40
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di Kotlin/dataframe
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 1 giorno
-
Add `columnOf(vararg DataFrame<T: DataRowSchema>): FrameColumn<T>` overload for compatibilityApertaAPI
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
Kotlin/dataframe#2202 · 3 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
documentation
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
Kotlin/dataframe#2195 · 2 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
AX
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
I maintainer di solito rispondono entro 1 giorno
-
Add minimal KDocs for `print` APIsForse già presa @zaleslaw l’ha presa 3 giorni fa. ApertaKDocs
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di Kotlin/dataframe
Issue simili
-
feat: 插入提示词时默认追加到输入框,长按才替换Apertaenhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
liberation-data/drivine-cypher-spec#67 ·
I maintainer di solito rispondono entro 1 giorno
-
feature
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
-
documentation good first issue help wanted P2 roadmap-v4
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
maziyarpanahi/openmed#5425 ·
I maintainer di solito rispondono entro 3 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
OneBusAway/vehicle-positions#250 ·
I maintainer di solito rispondono entro 1 giorno