I'd say it's even more of a dark pattern than that. They didn't encourage me to "upload my contact list" but rather "give access to my contacts" (or something like that) Perhaps the difference is trivial in how it's coded yet even though I've removed their access to my contacts, they still have my contacts. I think they should have to delete them whenever I remove their access, or not even upload them in the first place but just read them when necessary.
Also, some apps seem to do this with photos, asking for access, does anyone know if these apps also upload all of one's photos once the user grants permission on iOS?