There is a lot more that can hurt quality of life than something as obvious and severe as imprisoning someone. And if your government has the legal authority to torture you, you need a new government.
Failing to control sensitive personal information, and for that matter the inevitable mistaken information that will go along with it, could harm innocent people for reasons including but certainly not limited to: their religious views, their political inclination, their stance on controversial subjects such as abortion or legalisation of drug use, their employment history, any previous criminal activity no matter how minor and how long ago, and the lies a bitter ex once told about them in an online forum.
The kinds of harm caused might include but again are certainly not limited to: inability to get various kinds of insurance or paying excessive premiums, inability to get a job or to negotiate a fair employment contract and compensation when they do, inability to get credit, inability to travel by certain modes of transport, inability to attend certain public events, inability to send their kids to a good school, inability to meet the special someone they would have had those kids with, and in too many real world examples already, harassment, assault, injury, or death.
Governments should be restricted in the personal information they collect and how they can use it, not least because "government" is a sweeping term that probably includes a substantial proportion of the entire adult population in any first world country. But in some respects, preventing the unjustified collection, processing and disclosure of personal information in the private sector is far more important, because that's where most of the risks of nasty but not life-destroying, hire-a-lawyer-and-sue-for-millions damage will occur.
