I did the exact opposite, I got tired of vanilla Emacs and managing my custom configuration. Then I found about Spacemacs and never looked back.
I didn't get any strange bugs or slowdowns, but then I am using the `develop` branch which is going to be the upcoming .3 release. I also have a few dozen layers installed.
I would miss Spacemacs' layers way too much if I reverted to vanilla. Its integration of evil-mode is also fantastic, something I don't want or need to recreate.