The restrictions on Nazi symbolism are clearly infringing on freedom of speech, and should be recognized as such.
On the other hand, they are somewhat limited in scope (though not enough to my taste, see video games) and more importantly, understandable in their very specific context. They were arguably needed in the aftermath of WW2 when denazification didn't go as smoothly and naturally as could have been hoped, if only to remove the social proof that Nazism might not have been entirely defeated.