First you say it's really important to prosecute all crime because justice is about the response to crime.

Then you say it's silly to be worried that a bunch of "small crime" (furthermore, there's nothing to indicate in this case that this is a small crime) goes unpunished all the time.

Which one is it? Do we care about crime or don't we? I'd say it's actually the little crimes going unpunished that worry me the most... car theft, shoplifting, etc. These signal to participants that it's okay to behave in a way that is not in line with the stated laws of the land. Building this safe space for petty crime is far more dangerous than having a one-off corrupt asshole who committed a more "serious" crime run free on a legal technicality, because the safe-space normalizes bad behavior and desensitizes society to crime.