Dataset Finder (Data Discovery Tool)

What

Data analysis and exploration typically begins with the assumption that the right dataset exists. For many scenarios, this assumption holds (e.g., organizational data already exists is a tidy csv or json file). However, for other use cases, the right dataset may not exist and needs to be found.

The high level goal of this functionality is

provide a set of approaches to finding data given some query or representation of the user's intent.

How

Supported approaches may include the following:

• Heuristic strategy: define a work flow for identifying datasets that may be relevant. For example, support fixed providers like
  • data.gov
  • GHO https://www.who.int/data/gho/info/gho-odata-api
  • github to find csvs, or json files relevant to queries.
• Live agent strategy: define some mechanism that leverages web search in identifying related relevant datasets.

Possibly start off with a a base DataFinder class (find method), HeuristicsDataFinder subclass, AgentDataFinder subclass.

p.s. if you are interested in working on this, please share thoughts on your general approach for discussion and comment.