It's expensive in time and money to stop and start the trains. Also you need to maintain a station and employ people there.
If you want to provide for those with physical difficulties where walking 4 blocks is an issue, you support those people directly by, say, paying for their Ubers/cabs.
Are you American by any chance? I find Americans are more prone to this particular issue of not being able to make exceptions where necessary. People can't walk 4 blocks? Make everyone wait! Have more stops! Build more stations!
I mean this is exactly why Prop 13 in California was and is ridiculous. If we're really concerned about old people being forced out of their homes by rising property tax bills, why not just give them in particular a subsidized rate? Why do houses in Malibu and Beverly Hills need caps on property taxes because of some retired person in Bakersfield?