pyg-team/pytorch_geometric

[Roadmap] Heterogeneous Graphs Explainability Support

Open

#9,112 opened on Mar 26, 2024

View on GitHub
聽(11 comments)聽(0 reactions)聽(1 assignee)Python聽(19,985 stars)聽(3,514 forks)batch import
0 - Priority P0featuregood first issuehelp wantedroadmap

Description

馃殌 The feature, motivation and pitch

Explainability is a key feature of GNNs, which is already implemented in PyG. However, of all the features introduced, only a few have been adapted to heterogeneous graphs.

Algorithms: Of all the algorithms implemented for explainability, only the Captum algorithm is compatible with heterogeneous graphs. It would be interesting to adapt other specific algorithms for graphs such as GNNExplainer or PGExplainer, and other algorithms such as AttentionExplainer.

Moreover, the algorithms adapted by PyG could be extended with new algorithms that have been published over the years (for example, see this survey), but this isn't just for heterogeneous graphs. Maybe in the future, we could work on creating new algorithms simultaneously for heterogeneous graphs, without this gap.

  • Adaptation of GNNExplainer for heterogeneous graphs.
  • Adaptation of PGExplainer for heterogeneous graphs.
  • Adaptation of AttentionExplainer for heterogeneous graphs.
  • Implementation of new explainability algorithms. System to allow adaptation to heterogeneous graphs without additional work.

Features: Some features available in the explanations of homogeneous GNNs are missing in heterogeneous GNNs. For example, the visualize_graph method of Explanation is not available for HeteroExplanation. This right now can be done with the get_explanation_subgraph method, and generating the plot by hand with NetworkX, but it would be nice to implement it to do it automatically.

  • Implementation of missing features for HeteroExplanation, such as visualize_graph.

Metrics: Currently, the available metrics such as Fidelity or Faithfulness are only available for homogeneous graphs, but those metrics could be adapted for heterogeneous graphs. To continue the work of #5628, we could think implementing new metrics for all kind of graphs, such as sparsity or stability (e.g. see https://arxiv.org/pdf/2012.15445.pdf)

  • Adaptation of fidelity, faithfulness and characterization score for heterogeneous graphs.
  • Sparsity metric for all graphs.
  • Stability metric for all graphs.

Alternatives

No response

Additional context

No response

Contributor guide

Tech stack
python
Domain
machine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
PyTorch GeometricGraph Neural NetworksExplainability algorithmsPython
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
10
Research direction
Start by examining the existing homogeneous explainability implementations in PyG (e.g., GNNExplainer, PGExplainer) in the `torch geometric/explain/` directory. The next step is to adapt these algorithms to work with heterogeneous graphs, following the patterns used for CaptumExplainer. Additionally, implement missing features like `visualize graph` for `HeteroExplanation`. For metrics, extend Fidelity and Faithfulness for heterogeneous graphs and consider implementing sparsity and stability metrics as described in the linked paper.
[Roadmap] Heterogeneous Graphs Explainability Support 路 pyg-team/pytorch_geometric#9112 | Good First Issue