OpenRefine/OpenRefine

Bug found when applying reconciliation operations

Open

#3.829 aberto em 19 de abr. de 2021

Ver no GitHub
 (11 comments) (0 reactions) (1 assignee)Java (1.891 forks)batch import
Difficulty: IntermediateType: Bughelp wantedreconciliationundo/redo/history

Métricas do repositório

Stars
 (10.056 stars)
Métricas de merge de PR
 (Mesclagem média 4d 18h) (10 fundiu PRs em 30d)

Description

I am not sure if that's a bug or not, but the behavior is very weird. This is happening only when reconciling a dataset

  • from a blank OpenRefine instance
  • using undo/redo, apply

To Reproduce

You can try on branch pull/3777/head:issue-3424-recon

You need reconcile-csv up and running

/usr/bin/java -Xmx2g -jar reconcile-csv-0.1.2.jar  ./main/tests/cypress/cypress/fixtures/csv-reconcile-species.csv scientific_name taxon_id
  • delete your OpenRefine working directory
  • start open refine
  • create a project by copy pasting the following dataset
record_id,date,location,species
1,2017-06-23,Maryland,Hypsibius dujardini
2,2018-06-09,South Carolina,Protula bispiralis
3,2018-06-09,West Virginia,Monstera deliciosa
15,2018-09-06,West Virginia,Bos taurus
16,2017-10-05,Maryland,Amandinea devilliersiana
24,2015-05-01,West Virginia,Faciolela oxyrynca

  • apply the following operations
[
  {
    "op": "core/recon",
    "engineConfig": {
      "facets": [],
      "mode": "row-based"
    },
    "columnName": "species",
    "config": {
      "mode": "standard-service",
      "service": "http://localhost:8000/reconcile",
      "identifierSpace": "http://localhost:8000/",
      "schemaSpace": "http://localhost:8000/",
      "type": {
        "id": "/csv-recon",
        "name": "CSV-recon"
      },
      "autoMatch": false,
      "columnDetails": [],
      "limit": 0
    },
    "description": "Reconcile cells in column species to type /csv-recon"
  }
]

Current Results

The grid is not in the same state than when reconciling "manually", using standard UX panels

See, the original grid, from where I extracted the operations Screen Shot 2021-04-19 at 08 20 28

The grid when applying the operations (starting from a fresh OpenRefine instance, with an empty working dir)

Screen Shot 2021-04-19 at 08 19 26

The grid doesn't show neither "search for match", or "Choose new match", so we can't be sure of the state for each row, reconciled, matched or not?

Additional bug or potential bug found:

At first, I was thinking it was because the CSV service was not registered. I've added a line in the Cypress test to set a preference reconciliation.standardServices, with the following value

[
   {
      "name":"CSV Reconciliation service",
      "identifierSpace":"http://localhost:8000/",
      "schemaSpace":"http://localhost:8000/",
      "defaultTypes":[
         
      ],
      "view":{
         "url":"http://localhost:8000/view/{{id}}"
      },
      "preview":{
         "width":500,
         "url":"http://localhost:8000/view/{{id}}",
         "height":350
      },
      "suggest":{
         "entity":{
            "service_url":"http://localhost:8000",
            "service_path":"/suggest",
            "flyout_service_url":"http://localhost:8000",
            "flyout_sercice_path":"/flyout"
         }
      },
      "url":"http://localhost:8000/reconcile",
      "ui":{
         "handler":"ReconStandardServicePanel",
         "access":"jsonp"
      }
   }
]

It's partially working, and the grid ends-up in a new state, in-between what I was expecting and the original grid Notice the "search for match" that is now showing

Screen Shot 2021-04-19 at 08 30 58

The problem is that it takes 3 or 4 attempts for the test to pass, the first times, the grid doesn't show "Search for match", same as the first bug spotted in this issue

Is the process to register a reconciliation service async ? That would explain why the items are not matched for the first attemps

Expected Behavior

The grid to be in the exact same state than when doing those steps with the regular user interface. The grid to show either "Search for match" or "Choose new match"

Versions

OpenRefine Version 3.5-beta [68b148d]

Datasets

1618812648326.csv

Additional context

The goal of this was to restore a reconciled grid. Some tests requires reconciliation, but doing it with the user interface makes the test unreadable (in the code and in cypress viewer), so I was trying to use undo/redo to have one single API call to setup the initial conditions for a test to run. Goal was to test:

  • discard recon judgments
  • match each cell to best candidate

Guia do colaborador