Ah, yes, because it's impossible to store a contact list client-side. Or to encrypt arbitrary data like that to store server-side! Glad Moxie's looking out for us.
Take your SIM and install it in a clean phone. Install WhatsApp and verify the account.
Notice that when you open WhatsApp, only your group memberships are populated. Individual contacts are not, because they are held on the original device.