alibaba/GraphScope

Add more popular datasets to graphscope built-in datasets

Open

#1.015 geöffnet am 17. Nov. 2021

Auf GitHub ansehen
 (0 Kommentare) (0 Reaktionen) (1 zugewiesene Person)HTML (2.401 Stars) (301 Forks)batch import
good first issue

Beschreibung

We have several built-in datasets that can be easily loaded in one-line, located in the dataset directory of Aliyun OSS bucket graphscope, and the corresponding utility function to load them, located in python/graphscope/dataset/. We are planning to enrich the datasets continuously.

There's the procedure to add new datasets:

  1. Find a popular and appropriate dataset, adapt the format to property graph if necessary,
  2. Put all data files inside a folder, give the folder a meaningful name,
  3. Compress the folder, then upload the compressed file together with the original folder to the dataset folder of the OSS bucket. Assume you have a folder named foo/, and two files foo/nodes.csv and foo/edge.csv, after this step, you will have the following file structure in the bucket:
dataset
|-- foo.tar.gz
|-- foo
    |-- nodes.csv
    |-- edge.csv
  1. Write the loading function load_foo in a new file named python/graphscope/dataset/foo.py.
  2. A corresponding unit test is appreciated!

Contributor Guide

Tech Stack
python
Domain
data
Issue Type
feature
SchwierigkeitGeschätzte Implementierungsschwierigkeit für neue Contributors, von 1 für sehr kleine Änderungen bis 5 für Expertenarbeit.
3
Geschätzte ZeitEin grober Zeitrahmen, um zu recherchieren, zu implementieren, zu testen und einen Pull Request vorzubereiten.
1-2 days
AktivitätsstatusWie verfügbar das Issue gerade wirkt: frisch, aktiv, stale, blockiert oder wartet auf Maintainer-Input.
stale
KlarheitWie klar das Issue die erwartete Änderung, Akzeptanzkriterien und den nächsten Schritt erklärt.
clear
Voraussetzungen
basic Pythonfamiliarity with graph datasets
EinsteigerfreundlichkeitEin Score von 1 bis 100, der schätzt, wie zugänglich dieses Issue für First-Time Contributors ist.
75
Research-Richtung
The issue provides a clear step by step procedure for adding a new dataset. First, find a popular graph dataset and adapt it to property graph format if necessary. Then, create a folder with the dataset name containing CSV or other files, compress it as .tar.gz, and upload both the compressed file and the original folder to the dataset folder in the Aliyun OSS bucket. Next, write a Python loading function in a new file under python/graphscope/dataset/, following the pattern of existing loaders. Finally, add a unit test for the new loader. Look at existing dataset loaders in the repository, such as those in the same directory, for reference.