There's even more to emacs than to vim, if you start using emacs for more than just editing, doing things that have no equivalent in vim (like reading email, browsing the web, using IRC, or reading RSS news feeds, etc). For every one of these extra things, there's new stuff to learn, and more time to spend configuring emacs first to just work with those things and then to customize it the way you like.
For me the time investment is well worth it, but there's no denying that it takes a lot of time. A lot of time.