Yes. Try git or mercurial (AKA "hg":
http://selenic.com/mercurial/) rather than svn; both are much easier to set up and try out. When it only takes a second to set them up, you'll be more likely to experiment with them and discover what they're good at. SVN really doesn't have any real advantage over them these days.
There is a function in git and hg (an extension) called "shelve" which does exactly what you want - you can temporarily set aside the new feature in progress, commit the bugfix, and then go back to working on the feature. (If you knew ahead of time you were going to need to do this, it would probably have made more sense to do the feature in a branch, of course.)
People will argue back and forth whether git or hg is better, but in my experience they do the same things* , they just have different personalities. Pick one, get the hang of using it, and then experiment with the other - you'll be better able to make an informed decision. Both are significantly better than just about every other VC system.
* The advanced features they don't have in common are probably not going to matter until you know all the base functionality, and any new features in either will probably get ported to the other by then.
