I absolutely agree with you in substance. My main point is I think jailing is way overused and of usually of dubious value. It's extremely expensive, harms the victims (they literally have to pay taxes to pay for the aggressors), of dubious value in most cases, functions as criminal university, does little rehabilitation.
Of course it needn't be a centralized state per se. Somalis for example use 'xeer' law which is a scalable legal system that starts peer to peer and appealable upwards, mostly based on restitution/fines and ostracizing those who do not pay (eventually to the point, they could become 'outlaws' that have no protection from crime themselves).
I think restitution based legal system is ideal, but of course that would flip on its head the current system where the state ousts the victim and becomes the victim themselves and deprives the victim of restitution instead turning it into a big cronyism money making machine for themselves at the expense of everyone else. It would also mean the end of most 'victimless' crime, and god knows the wailing and nashing of teeth that would come when you couldn't prosecute someone for smoking a left handed cigarette because there is no victim to prosecute the case [or on behalf of].