Determine timezone of JDBC connections automatically on incremental ingestion
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 43/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Domain
- data-engineering
Research direction
Start by tracing incremental ingestion setup for datetime offset columns and the JDBC source configuration that reads server.timezone; inspect the connection and configuration entry points involved. Determine whether the JDBC connection can provide its timezone through an SQL statement, then compare the result with the configured timezone behavior. Done means incremental ingestion can determine and use the endpoint timezone automatically.
Written by the indexing model from the issue text.
Description
Background
When using incremental ingestion with 'datetime' columns it is important that the JDBC endpoint timezone is taken into account when setting up a datetime column as an offset column for incremental ingestion.
While it is possible to setup the timezone as part of Pramen JDBC source configuration (server.timezone), it would be great if this could be determined automatically using an SQL statement with the JDBC connection.
Feature
Determine timezone of JDBC connections automatically on incremental ingestion.
Example [Optional]
--
Proposed Solution [Optional]
--
- Dominant language
- Scala
- Stars
- 33
- Forks
- 5
- Avg merge
- 6h
- Merged PRs (30d)
- 15
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from AbsaOSS/pramen
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 28/100
Maintainers usually reply within 1 day
-
Add a parallelism mode that allows parallel job runs but never the same job for different info datesOpenenhancement
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
Jdbc Count doesn't use additional options from JDBC configPossibly taken @qwerty44-U claimed this 117 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Add ability to track number of records deviation and report it as warnings in email notificationsOpenenhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
Similar issues
-
area:scale audit priority:P2
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
KD5UJC-XYZ/scarab#231 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Maintainers usually reply within 2 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
riscv-boom/riscv-boom#819 ·