pyg-team/pytorch_geometric

Laplacian Eigenvector PE is not deterministic

Open

#7,499 opened on Jun 2, 2023

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Python (19,985 stars) (3,514 forks)batch import
bugdeterminismhelp wantedtransform

Description

Laplacian Eigenvector PE is not deterministic, i.e. it produces different encodings when applied to the same graph at different times even with everything seeded.

This is a code that reproduces it:

import torch
import random
import numpy as np
from torch_geometric.datasets import Planetoid
from torch_geometric.transforms import AddLaplacianEigenvectorPE

def seed_everything(seed: int):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)

seed_everything(42)
dataset = Planetoid(root='../datasets', name='Cora')
data = dataset[0]

lpe1 = AddLaplacianEigenvectorPE(k=124, attr_name="lpe1")
lpe2 = AddLaplacianEigenvectorPE(k=124, attr_name="lpe2")

# add lpe1
data = lpe1(data)
# add lpe2
data = lpe2(data)

# get pe
pe1 = data.lpe1
pe2 = data.lpe2

print(torch.allclose(pe1, pe2))  # prints False

# print for manual inspection
print(pe1[:5, :10])
print(pe2[:5, :10])

Environment

  • PyG version: 2.3.1
  • PyTorch version: 2.0.1+cu118
  • OS: Linux (Ubuntu 20.04)
  • Python version: 3.10
  • CUDA/cuDNN version: 12.0
  • How you installed PyTorch and PyG (conda, pip, source): pip.
  • Any other relevant information (e.g., version of torch-scatter): None.

Contributor guide

Tech stack
pythonpytorch
Domain
datamachine learning
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
PyTorchPyTorch Geometricunderstanding of seedingeigenvalue decomposition basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
55
Research direction
The issue is about non deterministic behavior in the AddLaplacianEigenvectorPE transform. The reproduction code shows that applying the transform twice with the same seed yields different results. The fix likely involves ensuring that all random operations in the eigen decomposition are seeded. Look at the implementation in torch geometric/transforms/add laplacian eigenvector pe.py. Consider using torch.use deterministic algorithms(True) or setting a seed for the underlying operation. Also check if the issue is related to the use of torch.linalg.eigh which may not be deterministic across runs. A possible approach is to set a manual seed within the transform or to use a deterministic version of the function.