I don't care to belong to any club that will have me as a member
:)
What would probably happen is that unless a strong initial group existed, seeding the website with good content, the site would die.
I get it you are not satisfied with HN sometimes, I know, me too. And it's happening more frequently. But the solution is not to open a club where only the cool guys will hang out. Because the cool guys are so demanded that they will choose where they will go. You have to offer more than a maybe it will, maybe it will not happen good experiences for the cool guys.
Now, if you had an stablished place, inviting good people to go would be easier. And you avoid the chicken and egg problem by working your ass off to provide the community with the seeds of how you want the interaction to happen. The page layout, the design, if threads or replies or initial up/downvotes, all of this will shape the initial user behaviour and might dictate what happens next (might... sometimes you don't have control).
Anyway. Invitations == bad, unless the person wants to be invited. Which is something hard to achieve :)