As the person that hid those upvote/downvote counts, I definitely remember that uproar well.
A relatively small group "migrated" for a few days, but didn't stay. Here's Whoaverse one month after the up/down counts were removed from Reddit (notice that they added visible vote counts, which they didn't have before): http://web.archive.org/web/20140718134533/http://whoaverse.c...
Other than the stickied site announcement, almost all of the posts only have a handful of votes and only a few have any comments.
The banned users were the first group that actually stuck around on Whoaverse/Voat, because they didn't have the option of just going back to Reddit.