pyg-team/pytorch_geometric

Implementation of LLM-to-LM GNN

Open

#9,361 opened on May 26, 2024

View on GitHub
聽(3 comments)聽(0 reactions)聽(1 assignee)Python聽(19,985 stars)聽(3,514 forks)batch import
1 - Priority P1examplefeaturehelp wanted

Description

馃殌 The feature, motivation and pitch

Paper: Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text Attributed Graph Representation Learning (ICLR 2024)

The paper uses the power of an LLM to get a meaningful explanation, which is then used to fine-tune a smaller LM+GNN. Really interesting paper. Would be great to have an example of it in PyG :)

Contributor guide

Tech stack
python
Domain
machine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
PyTorch Geometric basicsGraph Neural NetworksUnderstanding of LLMsPaper reading comprehension
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Study the paper 'Harnessing Explanations: LLM to LM Interpreter for Enhanced Text Attributed Graph Representation Learning' from ICLR 2024. The implementation requires combining an LLM (e.g., from Hugging Face transformers) with a GNN model in PyTorch Geometric. Follow the architecture described in Section 3. Review existing PyG examples for reference, such as 'examples/ogbn arxiv.py' for node classification. Implement a new module 'LLMtoLMGNN' in torch geometric.nn. The feature is not yet assigned, so contributions are welcome. Ensure to include proper documentation and tests.
Implementation of LLM-to-LM GNN 路 pyg-team/pytorch_geometric#9361 | Good First Issue