The chief problem with this (that Knockout still hasn't solved to this day that I know of) is progressive enhancement. Time and time again I try Knockout but I can never get it to progressively enhance based on existing data on the page, because of how overly declarative it is within the HTML elements themselves.
I don't want to go so far as to say that it can't be done, but it's certainly not nearly as easy as my front-end framework of choice right now (Backbone). That's pretty much a deal-breaker right there, and should be for most people wanting to design accessible web applications (which should be nearly everyone).