Multi-output multi-lengthscale GPs [feature request] [discussion]

Hey everyone,

I have coded up a multi-output multi-lengthscale GP (i.e. separate kernel hyperparameter per input dimension and per output dimension). It is pretty efficient, and scalable combined with inducing points. I took the code from GP contrib and modified it for research purposes (neural data analysis for big data), but if there is enough interest here it should be pretty easy to put this into Pyro itself. The question I have is, how should I package it in? Replace the current GP, or leave the option to use simple GP versus multi-dimension GP, or provide a separate subpackage in addition to contrib.GP?

I am aware of the interfacing with GPyTorch. Initially I tried to use that, but it is not straightforward to perform all PPL features (the loss objective is limited in this interface, GPLVM is tricky etc.).