microsoft/lida

Dataset Finder (Data Discovery Tool)

Open

#10 aberto em 21 de ago. de 2023

Ver no GitHub
 (3 comments) (0 reactions) (0 assignees)Jupyter Notebook (378 forks)github user discovery
good first issue

Métricas do repositório

Stars
 (3.253 stars)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

What

Data analysis and exploration typically begins with the assumption that the right dataset exists. For many scenarios, this assumption holds (e.g., organizational data already exists is a tidy csv or json file). However, for other use cases, the right dataset may not exist and needs to be found.

The high level goal of this functionality is

provide a set of approaches to finding data given some query or representation of the user's intent.

How

Supported approaches may include the following:

  • Heuristic strategy: define a work flow for identifying datasets that may be relevant. For example, support fixed providers like
  • Live agent strategy: define some mechanism that leverages web search in identifying related relevant datasets.

Possibly start off with a a base DataFinder class (find method), HeuristicsDataFinder subclass, AgentDataFinder subclass.

p.s. if you are interested in working on this, please share thoughts on your general approach for discussion and comment.

Guia do colaborador