Automatize as many checks as possible from the instance/ solution
#582 aperta il 5 dic 2024
Metriche repository
- Star
- (21 stelle)
- Metriche merge PR
- (Metriche PR in attesa)
Descrizione
Is your feature request related to a problem? Please describe. I'm always frustrated when... I have to re-do the same tests on each problem.
Describe the solution you'd like An easy way to give the primary-foreign key relationships between tables to validate that. In code or somewhere else.
Describe alternatives you've considered include it in the json-schema. But I prefer to include it as code because I always forget to update the json-schema.
Additional context
As inspiration, below is an example of how I did it in another project.
There, the relationships were defined with a dictionary of dictionaries.
For example, the suppliers table with primary key= supplier was to be checked with the columns named supplier in the tables: suppliers_c, supliers_t and sourcing_denied.
Another example, the nodes table with key=node was to be checked with the column named node1 in the transport table, the column node2 in the transport table, the column node from the nodes_cap table, and so on.
The error includes the pair of tables names, the pair of column names and the actual key that was not in the master.
def check_masters(self) -> SuperDict:
tables = dict(
suppliers=dict(
key="supplier", tables=["suppliers_c", "suppliers_t", "sourcing_denied"]
),
countries=dict(
key="country",
tables=[
"countries_t",
"countries_c",
"sourcing_denied",
"countries_c_eq",
],
),
nodes=dict(
key="node",
tables=[
("transport", "node1"),
("transport", "node2"),
"nodes_cap",
"nodes_cap_t",
# ("supply", "node1"), # this one can be empty / we do not care
("supply", "node2"),
],
),
commodities=dict(
key="commodity",
tables=["countries_t", "suppliers_c", "countries_c", "supply"],
),
)
err = SuperDict()
for table, el in tables.items():
key = el["key"]
_list_tables = el["tables"]
keys = self.data[table].keys()
for table2 in _list_tables:
key2 = key
if isinstance(table2, tuple):
table2, key2 = table2
distinct = self.data[table2].values_tl().take(key2).to_set()
for el2 in distinct - keys:
err[table, table2, key, key2, el2] = 1
return err