baobabsoluciones/cornflow

Automatize as many checks as possible from the instance/ solution

Open

#582 opened on Dec 5, 2024

 (2 comments) (0 reactions) (1 assignee)Python (5 forks)auto 404
clientdagsenhancementgood first issue

Repository metrics

Stars
 (21 stars)
PR merge metrics
 (PR metrics pending)

Description

Is your feature request related to a problem? Please describe. I'm always frustrated when... I have to re-do the same tests on each problem.

Describe the solution you'd like An easy way to give the primary-foreign key relationships between tables to validate that. In code or somewhere else.

Describe alternatives you've considered include it in the json-schema. But I prefer to include it as code because I always forget to update the json-schema.

Additional context As inspiration, below is an example of how I did it in another project. There, the relationships were defined with a dictionary of dictionaries. For example, the suppliers table with primary key= supplier was to be checked with the columns named supplier in the tables: suppliers_c, supliers_t and sourcing_denied.

Another example, the nodes table with key=node was to be checked with the column named node1 in the transport table, the column node2 in the transport table, the column node from the nodes_cap table, and so on. The error includes the pair of tables names, the pair of column names and the actual key that was not in the master.

    def check_masters(self) -> SuperDict:
        tables = dict(
            suppliers=dict(
                key="supplier", tables=["suppliers_c", "suppliers_t", "sourcing_denied"]
            ),
            countries=dict(
                key="country",
                tables=[
                    "countries_t",
                    "countries_c",
                    "sourcing_denied",
                    "countries_c_eq",
                ],
            ),
            nodes=dict(
                key="node",
                tables=[
                    ("transport", "node1"),
                    ("transport", "node2"),
                    "nodes_cap",
                    "nodes_cap_t",
                    # ("supply", "node1"),  # this one can be empty / we do not care
                    ("supply", "node2"),
                ],
            ),
            commodities=dict(
                key="commodity",
                tables=["countries_t", "suppliers_c", "countries_c", "supply"],
            ),
        )
        err = SuperDict()
        for table, el in tables.items():
            key = el["key"]
            _list_tables = el["tables"]
            keys = self.data[table].keys()
            for table2 in _list_tables:
                key2 = key
                if isinstance(table2, tuple):
                    table2, key2 = table2
                distinct = self.data[table2].values_tl().take(key2).to_set()
                for el2 in distinct - keys:
                    err[table, table2, key, key2, el2] = 1
        return err

Contributor guide