DP+ 3.0 tracking issue

オープン
  • 何か問題がありますか？
#259 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
8/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
静か
技術スタック
bootstrap, postgresql, python
領域
documentation

調査の方向性

This is a tracking issue for the DP+ 3.0 release rather than a single task, and its unchecked item (incorporating the TX Form Pages: Registration, Upload and Review formpages) is design-heavy. Start by reading the linked PRs #263, #264 and #307 to see how the work has landed, and check the scheming form snippets in the repository. Done would mean the remaining unchecked items are closed or split into their own issues.

索引モデルが issue の本文から書いたものです。

説明

DP+ 3.0 will make the Data Resource Upload Flow (DRUF) that was developed for the Texas Water Data Hub generally available.

The DRUF is interactive FAIRification, featuring:

  • ckanext-scheming 3.0 powered Dataset Form Pages with a standard Upload page in the beginning, and a Review page at the end.
  • metadata formulas - Excel like formula DSL using jinja2 that can calculate/suggest metadata. They can either be pre-calculated and assigned automatically or offered as suggestions.
  • Extensible formula helper library - since the formulas are written in jinja2, additional helpers can be easily added. The formula helpers will prioritize precalculating DCAT-US 3 recommended optional properties that are otherwise too hard to calculate manually (#218)
  • suggestion UX/UI - a Suggestion UI using Bootstrap popovers that show the Suggestion and the Formula.
  • apply scheming 3.0 changes
  • reorganize project for maintainability https://github.com/dathere/datapusher-plus/pull/264
    (DO THIS FIRST, BEFORE ITEMS BELOW)
  • https://github.com/dathere/datapusher-plus/pull/263 — merged 2025-11-23 (69f5481); truncate in its own txn so the AccessExclusive lock is held for the shortest possible window, eliminating UI hangs on large COPY-into-datastore jobs.
  • incorporate TX Form Pages implementation - specifically, the Registration Formpage and the Upload formpage in the beginning, and the Review Formpage at the end.
  • incorporate AI-powered suggestions for Description and Data Dictionary using qsv describegpt — backend in #301 (fdc3714), scheming UI in #302 (ad28ebc), qsv-flag + envelope-shape fixes from LM Studio E2E in #303, and Vitest + jsdom unit suite in #304 (which also caught a production bug where the polling JS was reading STATUS from the wrong path and would never have terminated). Opt-in via ckanext.datapusher_plus.enable_ai_suggestions (off by default; needs an OpenAI-compatible endpoint configured in qsv's describegpt prompt-file). Per-field opt-in via ai_suggestion: true on the scheming field config; AI buttons appear next to opted-in fields on markdown / text / select form snippets, polling JS reads suggestions from package["dpp_suggestions"]["ai_suggestions"] and applies on click.
  • extend qsv describegpt so that it can use a controlled vocabulary for suggested tags. The controlled vocabulary will be configurable and point to the specified CKAN resource with the following columns tag, description, AI guidance. AI guidance will be customizable instructions when/how to use the associated tag.
  • leverage qsv's delimiter inferencing (see #256) — done in efb254f; format_converter.py:319-340 now sniffs and normalizes non-comma CSVs
  • #261
  • #262
  • #265 — stash + restore Data Dictionary across DP+ job failures, landed in #307 (59e08d7). Analysis stage writes a small per-resource JSON snapshot to disk before deleting the existing datastore; _rollback_database reads it back on failure and re-creates the datastore with the original info dicts (and proper type derived from type_override) and zero rows. A retry branch in AnalysisStage._parse_stats re-loads the stash when no live datastore exists, so the dictionary survives even when the failure happened outside the database transaction. Stash mtime is surfaced in restore logs so operators can spot stale-restore vs. genuine retry. New knob ckanext.datapusher_plus.dictionary_stash_dir (defaults to <tempdir>/dpp_dict_stash).
主要言語
Python
スター
47
フォーク
33
平均マージ
2時間 28分
マージ済み PR（30日）
10

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

dathere/datapusher-plus のほかの issue

dathere/datapusher-plus の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。