OrionBelt converter drops a field's logical datatype in both directions

Open
#409 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend, data

Research direction

Start in ossie_to_obml.py and obml_to_ossie.py, focusing on _convert_field and _convert_column. Read tests/test_ossie_metric_no_silent_loss.py and run the suggested explicit datatype round-trip test. Done means spec datatype values map correctly in both directions and the canonical datatype field is emitted, while existing tests remain green.

Written by the indexing model from the issue text.

Description

Summary

Neither conversion direction carries a field's logical type through the
canonical spec datatype field:

  • Ossie->OBML ignores the declared datatype and derives OBML
    abstractType from a column-name heuristic, so a String field can be
    emitted as numeric and vice-versa (wrong type).
  • OBML->Ossie never writes the spec datatype field at all - the type is
    stashed only inside an OrionBelt custom_extensions blob (missing type).

A round-trip appears lossless only because the converter is self-consistent
with its own non-spec key; any other spec-compliant tool sees wrong or absent
types.

Root cause

Ossie->OBML (ossie_to_obml.py):

ossie_type = field.get("data_type", "")               # (1) wrong key
if ossie_type and ossie_type in OSSIE_TO_OBML_TYPE:    # (2) wrong-cased map
    abstract_type = OSSIE_TO_OBML_TYPE[ossie_type]
else:
    abstract_type = self._infer_obml_type(field)
  1. Wrong key. The spec field is datatype, no underscore, so the branch is dead
  2. Wrong-cased map. The spec enum is capitalized, but
    OSSIE_TO_OBML_TYPE is keyed lowercase with different terms.
    Even after fixing the key, case-normalizing is not enough:
    DateTime/DateTimeTz/Decimal/Float have no key - so case-normalizing is not enough

OBML->Ossie: canonical field never written (obml_to_ossie.py).
The type goes only into a vendor extension, under the non-spec key
data_type with a lowercase value; field["datatype"] is not set.

Illustrative Repro

# Ossie -> OBML: declared datatype ignored
for f, want in [({"name": "total_amount", "datatype": "String"},  "string"),
                ({"name": "customer_id",  "datatype": "Integer"}, "int")]:
    _, col = OssietoOBML(ossie={})._convert_field(dict(f))
    got = col["abstractType"]
    print(f"fwd {f['datatype']:8} -> {got:8} (want {want}) [{'OK' if got==want else 'BUG'}]")

# OBML -> Ossie: canonical datatype never emitted
field = OBMLtoOssie(obml={})._convert_column("customer_id", {"code": "customer_id",
                    "abstractType": "int"}, "Orders", {})
print("rev canonical 'datatype' present?", "datatype" in field, "[BUG]" if "datatype" not in field else "")

Why this survived

The existing fixtures (test_ossie_converter_vendors, test_ossie_metric_no_silent_loss)
use the same wrong key data_type, so the suite stays green with the bug present.

Drop-in test (currently fails). Suggested home:
tests/test_ossie_metric_no_silent_loss.py, which already guards this "no
silent loss on field conversion" path.

def test_explicit_datatype_roundtrips():
    _, col = OssietoOBML(ossie={})._convert_field({"name": "total_amount", "datatype": "String"})
    assert col["abstractType"] == "string"          # fwd: currently 'float'
    field = OBMLtoOssie(obml={})._convert_column("total_amount",
                {"code": "total_amount", "abstractType": "string"}, "Orders", {})
    assert field.get("datatype") == "String"        # rev: currently missing

Suggested fix

  • Ossie->OBML: read field.get("datatype") (fix the comment); rebuild
    OSSIE_TO_OBML_TYPE on the spec enum
  • OBML->Ossie: set the canonical field["datatype"] (capitalized enum value)
    in addition to any extension bookkeeping.
Dominant language
Python
Stars
2.2k
Forks
280
Avg merge
2d 7h
Merged PRs (30d)
32

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/ossie

All issues in apache/ossie

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.