If improving granularity is the goal, wouldn’t it make even more sense to have fractional votes? After all, there’s no real reason why each representative’s vote should count equally...
It's sort of nice to be able to reason about votes in a straightforward way, and you'd have even worse social dynamics if some members are literally more powerful inside the chamber than others.