FFmpegFrameGrabber processImage() method is very resource intensive

Converting from one color format to another is just generally very resource intensive. In my case most of the time the conversion is from yuv420p frames to rgb. The real question I have is whether it's possible to avoid this conversion and somehow directly display decoded frames. Is the gpu even capable of doing that? And even if it isn't I see that my processor is also getting quite a bit of work in when converting the frames so is it perhaps possible to leave the entire conversion up to the gpu?