What I think you're saying is that the reader should be customizable in terms of some DSL for a grammar. It would be nice, but I'm not sure how composable it would be (in the general case, I think it would come down to having the behavior of READ dependent upon a black-box
current-parser procedure).
The nice thing about readtables is that it exposes what in essence are transition hooks for each character, so you really don't need to care about the grammar of stuff you're not interested in parsing.
OTOH like you said, extending the syntax for numbers or symbols becomes quite hairy. But even with a DSL grammar approach, you'd need to change major parts of the grammar (which means copying and modifying the normative grammar of CL syntax - not that different from grabbing a portable CL reader implementation today (http://twitter.com/vsedach/status/26484049015)).
The big downside is deciding on which class of grammars that will support, and how they will be represented.