Packages can do weird things like auto-loading into the interpreter (example: [0]). So in a scenario where a malicious package has ended up on your machine, you're a bit screwed whether it's a .so or a .py. I believe that was the point OP was making -- a pure-Python wheel is not really any safer than a wheel with embedded binaries.
[0]: https://github.com/pyston/pyston/blob/1d65d4831912179c26bb27...