apache/sedona

feat: add to_sedonadb() method

Open

#2,511 建立於 2025年11月19日

在 GitHub 查看
 (1 留言) (0 反應) (0 負責人)Scala (1,953 star) (693 fork)batch import
help wanted

描述

It would be nice to have an interface that converts a SedonaSpark DataFrame to a SedonaDB DataFrame easily. Here is a current solution that works:

import sedona.db
sd = sedona.db.connect()

df = sd.create_data_frame(dataframe_to_arrow(spark_df))

This could be nice:

spark_df.to_sedonadb()

But maybe we'd have to do this:

spark_df.to_sedonadb(sd)

This would allow for cool spatial workflows, like this:

  • Read an Iceberg table with SedonaSpark and perform big data operations with a filtering operation at the end to make the data small enough to fit on a single machine
  • Convert the SedonaSpark DataFrame to SedonaDB
  • Use a library that's compatible with SedonaDB, like lonboard, to create a graph

Let me know what you think!

貢獻者指南

技術棧
python
領域
dataapi
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-3 hours
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
fresh
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
Basic knowledge of Apache SedonaPython programmingSpark DataFrame API
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
50
研究方向
First, locate the Python source code for Sedona's Spark DataFrame wrapper, likely in `python/sedona/core/`. Look for existing conversion methods like `to geopandas()` or `to spatial rdd()`. The new `to sedonadb()` method should be added to the same class. It must accept a SedonaDB connection object and internally call `dataframe to arrow(spark df)` and `sd.create data frame()`. Ensure the method follows existing patterns for type hints and documentation. Review the issue comment for any additional insights from maintainers.