intent: exports: fields have no text normalisation - per-file case, allowed characters, replacements and a refusal on information loss
Maintainers usually reply within 1 day
A pull request for this has already been merged.
- #7999 by @nedelcho-delchev-tues — merged
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
The issue names engine-intent unit tests and an export integration test, but no source files or entry points. Start by locating how exports: fields are parsed and written, then trace how export run records store field values. Done means each proposed option is covered by unit tests, onLoss rejects the run with record and field details, and the integration test verifies an uppercased, stripped name column in a delimited file.
Written by the indexing model from the issue text.
Description
Problem
An exports: field writes its source value verbatim. The field options are from / literal / param, width / align / pad / truncate, scale / decimalSeparator, pattern (dates) and the XML name / attribute — there is no way to normalise text for a target file format. The run only fails when a character cannot be encoded in the target charset.
Some regulator formats accept a far narrower character set than the master data should hold. Example: the Bulgarian NRA validators for the payroll Declarations обр. 1 / обр. 6 accept only uppercase Cyrillic, digits and - in names, while the VAT-ledger validator of the same agency accepts Latin, №, „“, &, /, ., , and more. The legal name must stay as registered in the master data; each file needs its own normalisation, in one declared place, not in a Java helper per module.
Proposal
Per-field text options on exports: fields:
case: upper | lowerallowed: "<character class>"— characters outside it are handled byreplace/stripreplace: { "№": "", "/": " " }— explicit replacements applied beforeallowedonDisallowed: strip | space | fail(defaultfail, keeping the "nothing is written approximately" rule);collapseSpaces: trueonLoss: fail— refuse the run (recorded FAILED, 422) naming the record and field when normalisation changed more than case, punctuation and whitespace — e.g. a Latin-script value written into a Cyrillic-only field — instead of writing a value nobody typed.
The run record keeps the original and the written value for every field that was normalised, so a rejected upload can be traced back to the record.
Verify
Unit tests in engine-intent for each option and the onLoss refusal; one export IT writing a delimited file with an uppercased/stripped name column.
- Dominant language
- Java
- Stars
- 278
- Forks
- 97
- Avg merge
- 18h 57m
- Merged PRs (30d)
- 249
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from eclipse-dirigible/dirigible
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
eclipse-dirigible/dirigible#7958 ·
Maintainers usually reply within 1 day
-
intent: a create-from without fromStatus answers 200 on a missing or unknown source id (no 400/404)Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
eclipse-dirigible/dirigible#7945 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
eclipse-dirigible/dirigible#7891 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
eclipse-dirigible/dirigible#6538 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
eclipse-dirigible/dirigible#4812 ·
Maintainers usually reply within 1 day
All issues in eclipse-dirigible/dirigible
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
checkstyle/checkstyle#22145 ·
Maintainers usually reply within 1 day
-
rset command throws NPE in ResultsFormat.guessSyntaxPossibly taken @Ostrzyciel claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
enhancement good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
objectionary/eo#9593 ·
Maintainers usually reply within 1 day
-
release bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
OpenLiberty/open-liberty#36124 ·
Maintainers usually reply within 1 day