Using `pyg` sampler causes lightning to skip validation step

Bug description

I am trying to use the dynamic batch sampler from torch_geometric (link) for my regression task on graphs. For some reason, using the batch sampler as follows skips the validation step. I think this is linked to this issue. This is how I am setting up my batch sampler,

from torch_geometric.loader import DataLoader
import pytorch_lightning as pl

class BaseDataModule(pl.LightningModule):
    # usual init, setup, ....

    def train_dataloader(self):
        batch_sampler = <instantiate_dynamic_batch_sampler>

        return DataLoader(
            self.train_dataset,
            batch_size=batch_size,
            shuffle=shuffle,
            batch_sampler=batch_sampler,
            **self.kwargs,
        )

    def val_dataloader(self):
        """Creates val dataloader"""
        batch_sampler = <instantiate_dynamic_batch_sampler>

        return DataLoader(
            self.val_dataset,
            batch_size=batch_size,
            shuffle=shuffle,
            batch_sampler=batch_sampler,
            **self.kwargs,
        )

What version are you seeing the problem on?

version 2.0.4

How to reproduce the bug

No response

Error messages and logs

# Error messages and logs here please

Environment

#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info

No response