I noticed that the interface was extremely laggy no matter what I did. Maxed out settings, installed guest additions ect. Nothing helped. I then found this article:
Apparently the issue is Unity, and there is no fix. After rolling back to Gnome following the instructions there, performance was fantastic.
Hopefully this helps someone else.