The problem is that, for this to be the case, constituents must have a working communication channel. But when you have a single representative ostensibly speaking for millions (or even tens of thousands) of people, any such channel is quickly overwhelmed, and becomes mostly a token thing.

Council democracy tries to solve it by ensuring that, at every level, each delegate speaks for a relatively small group of people - small enough that it can regularly assemble to keep the delegate under control at all times.