Add more popular datasets to graphscope built-in datasets · alibaba/GraphScope#1015

(0 comments) (0 reactions) (1 assignee)HTML (2,401 stars) (301 forks)batch import

good first issue

説明

We have several built-in datasets that can be easily loaded in one-line, located in the dataset directory of Aliyun OSS bucket graphscope, and the corresponding utility function to load them, located in python/graphscope/dataset/. We are planning to enrich the datasets continuously.

There's the procedure to add new datasets:

Find a popular and appropriate dataset, adapt the format to property graph if necessary,
Put all data files inside a folder, give the folder a meaningful name,
Compress the folder, then upload the compressed file together with the original folder to the dataset folder of the OSS bucket. Assume you have a folder named foo/, and two files foo/nodes.csv and foo/edge.csv, after this step, you will have the following file structure in the bucket:

dataset
|-- foo.tar.gz
|-- foo
    |-- nodes.csv
    |-- edge.csv

Write the loading function load_foo in a new file named python/graphscope/dataset/foo.py.
A corresponding unit test is appreciated!

コントリビューターガイド

技術スタック: python
領域: data
Issue 種別: feature
難度: 3
推定時間: 1-2 days
活動状況: stale
明確さ: clear
前提条件: basic Pythonfamiliarity with graph datasets
初心者向け度: 75
調査方針: The issue provides a clear step by step procedure for adding a new dataset. First, find a popular graph dataset and adapt it to property graph format if necessary. Then, create a folder with the dataset name containing CSV or other files, compress it as .tar.gz, and upload both the compressed file and the original folder to the dataset folder in the Aliyun OSS bucket. Next, write a Python loading function in a new file under python/graphscope/dataset/, following the pattern of existing loaders. Finally, add a unit test for the new loader. Look at existing dataset loaders in the repository, such as those in the same directory, for reference.