The assumption that UI design must be done by hand is probably one of the most important "bottleneck" in software development.
When you think about it, the job of a designer is often quite systematic. You have some entities/data you need to communicate to the user, through whatever interface/device that's available to him. A touchscreen, a keyboard, knobs, LEDs, microphone, speaker, paper, etc.
When your "user" is a computer, JSON (while not perfect) seems to do the job as an interface. In the case of humans, JSON does a poor job at efficiently communicating information.
For humans:
- A calendar is better than "2012-11-05"
- A color is better than "#FF3300"
- An image is better than "http://example.com/image.png"
- A clickable link is better than "http://example.com/document.html"
The list goes on. We can easily generate a basic UI based on complex entities, and map specific types to custom/reusable templates if needed.
Now that your UI can automatically be generated from data, you can build an app (business model) once, and make it usable (and actually look and feel good/native) on any device. A smartwatch, a smartphone, a smarttv, etc.
Basically, the core of software development should be knowledge representation. Describe the world semantically (with RDF or similar technologies), and let the UI-compiler generate a UI for any given target platform, language, culture, user preferences. That's what responsive design should be all about.