I don't think you've played minecraft and done modding.
If your server requires 1.6.4 and railcraft 1.2.3.4.5.6 well then anyone who connects requires MC 1.6.4 (free, no problem) and ALSO requires railcraft 1.2.3.4.5.6 and mods are stereotypically hosted on the scummiest pr0n-iest adfest drive by download sewers of the internet (sorry, but its true)
So say you want to host a server called "jiggy's minecraft". Simply require a mod to log into your server called jiggy-2014 for the entire year of 2014 and sell it in the in app store for "annual fee". Maybe drop the price by 1 / 12 every month thru the year. Guess the mod name you'll require in 2015, yes your mod will be called jiggy-2015 and be priced at your annual fee.
The mod might even provide something useful to the players. Maybe. This will probably kill/damage the ecosystem, maybe you'll never get to play railcraft, TE, and buildcraft on the same server ever again. That would suck. It is possible "real mods" won't be balkanized and only "subscription mods" will be balkanized.