Quantum computers break several security assumptions. But not all of them and we usually can replace the broken assumptions. Discovering that P=NP, or that one-way functions do not exist, on the other hand, would imply that several secure cryptographic constructions that we want to use are in fact impossible and would be a much scarier discovery.
Even if we focus on crypto only, from the perspective of people being oppressed around the world, it would be far better for them to be able to see what is being done by the powerful publicly rather than not. This is essential for transparency, which is essential for democracy.
I don't reject the ideals of e.g. https://en.wikipedia.org/wiki/A_Declaration_of_the_Independe..., but I believe that the technological solution through cryptography is not tenable, and if we got rid of cryptography, it would actually level the playing field.
Btw, if your polynomial algorithm for NP is any good, you should be able to break any encryption at all. The problem of breaking cryptographic systems is typically inside of both NP and co-NP. That intersection is suspected to be substantial smaller than NP by itself. (Of course, if it all collapses to P, that wouldn't make a difference.)
But regardless, it's important to realize that modern cryptography relies on a hypothesis. It might be effectively true for now, but it might not in the future.
> Btw, if your polynomial algorithm for NP is any good, you should be able to break any encryption at all.
In theory, yes, in practice, there is a pretty big difference between "I think I just discovered how to do Gaussian elimination to solve linear equations" and "I can routinely solve sparse linear systems with millions of variables". Historically, it wasn't done by a single individual in a span of couple years.
But here you say that: Security can depend on social contract and not technology. But the same argument can be applied to your original objection. We can mandate (in the social contract) that people have the right to privacy. (And there are some analogs where we do that, for example, we could have Gattaca-like dystopia where people's access to health care is based on genetics, yet all developed countries have a ban on such discrimination.)
I think it's far more dangerous that the government or adversary is technologically capable of being non-transparent than if ordinary people don't have capability to be non-transparent. It's because I believe power ultimately thrives on information asymetry, and the encryption only amplifies this asymetry.
I really like Assange's essay on the topic, where he describes that every big operation requires a sort of paper or document trail (which is used to coordinate many people), and this fact makes whistleblowing always an option.
However, I believe, encryption (and related things like trusted computing) helps to minimize the exposure of this trail even to internal actors, and in doing so, it makes whistleblowing more difficult.
Even ignoring that we don't currently have any usable quantum computers and there's no reason to believe they would be affordable, let alone cheap, this mean AES-256 is fine.
Eg if you were planning on breaking AES-128 by running 2^30 cores for 2^98 AES calls, it now only takes about 2^49 calls per core (2^79 total effort) plus the overhead of Grover's algorithm itself. There are also huge overheads from running everything on a quantum computer, some of which are theoretically avoidable (100x cost for error correction; gates take one clock cycle) and some of which are probably not (you must rewrite AES so that all computations are reversible). So breaking AES-128 might eventually be feasible, but AES-192 probably would not be.
There is a theoretical barrier against efficiently parallelizing Grover: all generic quantum brute-force algorithms (the kind that would work against a random function in a black box) require Omega(searchspace / depth) queries, at least for some model that may not quite match reality. (Edit: Omega and not O, since it's a lower bound)
Of course, AES isn't a random function in a black box, so there may be better attacks against it, but I'm not aware of any.