baobabsoluciones/cornflow

Automatize as many checks as possible from the instance/ solution

オープン

#582 opened on 2024/12/05

 (2 件のコメント) (0 件のリアクション) (1 人の担当者)Python (5 件のフォーク)auto 404
clientdagsenhancementgood first issue

Repository metrics

Stars
 (21 個のスター)
PR merge metrics
 (PR metrics pending)

説明

Is your feature request related to a problem? Please describe. I'm always frustrated when... I have to re-do the same tests on each problem.

Describe the solution you'd like An easy way to give the primary-foreign key relationships between tables to validate that. In code or somewhere else.

Describe alternatives you've considered include it in the json-schema. But I prefer to include it as code because I always forget to update the json-schema.

Additional context As inspiration, below is an example of how I did it in another project. There, the relationships were defined with a dictionary of dictionaries. For example, the suppliers table with primary key= supplier was to be checked with the columns named supplier in the tables: suppliers_c, supliers_t and sourcing_denied.

Another example, the nodes table with key=node was to be checked with the column named node1 in the transport table, the column node2 in the transport table, the column node from the nodes_cap table, and so on. The error includes the pair of tables names, the pair of column names and the actual key that was not in the master.

    def check_masters(self) -> SuperDict:
        tables = dict(
            suppliers=dict(
                key="supplier", tables=["suppliers_c", "suppliers_t", "sourcing_denied"]
            ),
            countries=dict(
                key="country",
                tables=[
                    "countries_t",
                    "countries_c",
                    "sourcing_denied",
                    "countries_c_eq",
                ],
            ),
            nodes=dict(
                key="node",
                tables=[
                    ("transport", "node1"),
                    ("transport", "node2"),
                    "nodes_cap",
                    "nodes_cap_t",
                    # ("supply", "node1"),  # this one can be empty / we do not care
                    ("supply", "node2"),
                ],
            ),
            commodities=dict(
                key="commodity",
                tables=["countries_t", "suppliers_c", "countries_c", "supply"],
            ),
        )
        err = SuperDict()
        for table, el in tables.items():
            key = el["key"]
            _list_tables = el["tables"]
            keys = self.data[table].keys()
            for table2 in _list_tables:
                key2 = key
                if isinstance(table2, tuple):
                    table2, key2 = table2
                distinct = self.data[table2].values_tl().take(key2).to_set()
                for el2 in distinct - keys:
                    err[table, table2, key, key2, el2] = 1
        return err

コントリビューターガイド