Add simple, self-contained SQL/DB examples for Kotlin DataFrame
维护者通常 1 天内回复
@zaleslaw 已经在做这个了。
开始于 2026年8月4日。
评估
这个 Issue 还没有评估数据。
描述
Motivation
Today the Database + Kotlin DataFrame (+ Kandy) integration is only demonstrated by the external
KotlinDataFrame-SQL-Examples project. Problems:
- its notebooks are mostly deprecated;
- the flagship example (IMDB) needs a >100 MB MariaDB dump downloaded from Google Drive — the barrier to
entry is so high that basically nobody actually runs it.
We want a set of small, self-contained examples inside the dataframe repository: no external dumps, no
mandatory Docker where avoidable, so that "clone → run → see a DataFrame from a database" just works.
What already exists (foundation to reuse)
- The
dataframe-jdbcmodule supports H2, SQLite, PostgreSQL, MySQL, MariaDB, MS SQL, DuckDB.
Entry points indataframe-jdbc/src/main/kotlin/.../io/readJdbc.kt:
readSqlTable,readSqlQuery,readResultSet,readAllSqlTables,
Connection.readDataFrame(...),DbConnectionConfig.readDataFrame(...),DataSource.readDataFrame(...). - "Postgres inside H2" is built in:
H2(mode = Mode.PostgreSql)(.../io/db/H2.kt,MODE=PostgreSQL) reads
with PostgreSQL logic. Reference tests:dataframe-jdbc/src/test/kotlin/.../io/h2/. - Testcontainers will be wired up soon:
testcontainersTest/localDbTesttasks indataframe-jdbc/build.gradle.kts,
image versions ingradle/libs.versions.toml(postgres18-alpine, mysql9.7, mariadb12.3.2). - Examples are standalone projects under
examples/projects/<name>/; all are automatically compiled during
:test(debug mode) viabuild-logic/.../dfbuild.buildExampleProjects.gradle.kts. Related existing ones:
exposed,hibernate(ORM↔DataFrame); CSV+Kandy inkotlin-dataframe-plugin-gradle-example. - Backend guide:
docs/StardustDocs/topics/guides/Guide-for-backend-SQL-developers.md.
Scope — new examples
Each example is a new folder under examples/projects/: a standalone Gradle project with its own
settings.gradle.kts, a tiny embedded/"fun" dataset, a README.md, and a runnable main(). DB drivers are
runtime dependencies. A companion Jupyter notebook may be added per example where it works cleanly, but the
Gradle project is the primary, CI-tested artifact.
-
sql-csv-join— join a CSV file with a database table.
readCsv(<resource>)+readSqlTable(...)from H2/SQLite →join+groupBy/aggregation.
Shows DataFrame as a bridge between data sources. Includes a Kandy chart of the result. -
sql-sqlite— read from a bundled.sqlitefile in resources. Zero setup.
Connection.readDataFrame("SELECT ...")/readSqlTable. Small themed dataset. -
sql-postgres— one database, two ways to run it:- zero-Docker:
H2(Mode.PostgreSql)withMODE=PostgreSQL, schema + data,readSqlQuery/readResultSet; - optional "real PostgreSQL" via Testcontainers (image already in the project) as a separate
main/README section.
Includes a Kandy chart.
- zero-Docker:
-
sql-ktor— a minimal Ktor server that reads a database into aDataFrameon
request and serves it over HTTP (JSON viadataframe-jsontoJson, and/or a small HTML page). Uses embedded H2
so it runs with no external setup.
Out of scope: a Spring Boot + full schema-migration example (too heavy — pulls in Spring, which the repo has
zero of today, plus a migration tool). Track it separately if wanted.
Datasets
Two viable sources: reuse datasets already bundled in the repo, or generate a tiny dataset in code.
Rule of thumb:
- Read-only "read from a real DB" demos → reuse a ready-made, recognizable dataset. The repo already ships
the Chinook SQLite DB atexamples/projects/exposed/src/main/resources/chinook.db(a digital-music store:
artists,albums,tracks,genres,invoices,customers— multiple joinable tables, well known,
~1 MB, no download). Best default. - Demos that must own their schema/seeding (Postgres types, Testcontainers) → generate a small themed dataset
in code via inlineCREATE TABLE+INSERT(idempotent, identical for H2-in-postgres-mode and a real
container, no binary blob to check in). Keep it small (~20–50 rows) and "fun". Pattern to copy:
createPostgresTestDataindataframe-jdbc/src/test/kotlin/.../io/postgresTestBase.kt.
Per example:
sql-sqlite→ reusechinook.dbas-is. Zero generation; run a query (e.g. top-selling genres, tracks
per album) viaConnection.readDataFrame("SELECT ..."). Strongest ready-made fit.sql-csv-join→ combine a real DB table (Chinookartists/albums) with a small hand-authored CSV
that joins on a key (e.g.artist,country,formedYear). Generate the tiny CSV side by hand so the join key and
result are obvious. Alternative fully-ready combo:movies.csv(already in themoviesexample) + a generated
in-memory ratings table.sql-postgres→ generate a small themed schema+rows in code (inline SQL). The same seeding runs against
H2(Mode.PostgreSql)and against the Testcontainers Postgres — no file dependency — and lets us show a couple
of Postgres-flavored types/queries deliberately.sql-ktor→ generate and seed an embedded H2 at startup (so the app is fully self-contained), or reuse
chinook.dbread-only. Serve the resultingDataFrameas JSON viadataframe-jsontoJson.
Notes: Chinook is a public, redistributable sample DB and is already committed, so no licensing/size concern.
Avoid the test-only safe_moz_places_sample.sqlite (Firefox history — not illustrative) except for smoke tests.
Implementation
- For each example create
examples/projects/<name>/following existing ones
(examples/projects/exposed/,.../kotlin-dataframe-plugin-gradle-example/): ownsettings.gradle.kts
(rootProject.name),build.gradle.kts,gradle/libs.versions.toml,src/main/kotlin, resources,README.md. - Mirror each example into
examples/projects/dev/<name>/(dev mirror targeting the master library version). - In
build-logic/.../dfbuild.buildExampleProjects.gradle.ktsadd the new libraries (ktor, postgres/sqlite
drivers, testcontainers; kandy already present) toversionsToSyncandexampleDependencyUpdatesso version
sync anddependencyUpdatesare aware of them. Add new versions to the rootgradle/libs.versions.toml. - Update
examples/README.md: a section for the SQL/DB examples with links; mark the old
KotlinDataFrame-SQL-Examplesas "advanced / large dataset". - (Optional) add companion notebooks under
examples/notebooks/<name>/where they run cleanly, and cross-link
fromdocs/.../Guide-for-backend-SQL-developers.md.
Acceptance criteria
./gradlew runBuildAllExampleFolders -Pkotlin.dataframe.debug=truecompiles all new examples (release + dev).- Each example runs locally with no external data download; the Testcontainers variant runs only when Docker is present.
- Every example has a
README.mdexplaining what it shows and how to run it. examples/README.mdis updated.
- 主要语言
- Kotlin
- 星标
- 1.1k
- 派生
- 87
- 平均合并
- 2 天 21 小时
- 30 天内合并 PR
- 40
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
Kotlin/dataframe 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
API
难度 2/5 1-3 小时 新手友好度 66/100
Kotlin/dataframe#2202 · 3 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
documentation
难度 2/5 1-3 小时 新手友好度 72/100
Kotlin/dataframe#2195 · 2 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
AX
难度 1/5 1 小时以内 新手友好度 90/100
维护者通常 1 天内回复
-
Add minimal KDocs for `print` APIs可能已有人在做 @zaleslaw 于 4 天前认领。 未关闭KDocs
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复
相似的 Issue
-
-priority- blocked good first issue help wanted
难度 2/5 1-3 小时 新手友好度 68/100
scribe-org/Scribe-Android#761 · 2 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
-
难度 1/5 1 小时以内 新手友好度 90/100
OneBusAway/vehicle-positions#288 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 90/100
-
难度 1/5 1-3 小时 新手友好度 82/100
LibChecker/LibChecker-Rules#1456 ·