You've misunderstood my point: I'm not talking about reducing the difficulty, I'm talking about (sub-linearly) increasing the mining reward as the difficulty grow.
Also the difficulty scaling is important even if you don't take security into account: if you don't raise it, then the block mining frequency rise progressively, and then you end up with more and more orphaned blocks, until you end up with parallel chains that don't really converge to a single coherent one.