I think it's just that they're tricky when it comes to ownership. People who write code that depends on some type of file or directory ownership for security often don't think about the ways symlinks can be used to bypass their security model.

You can sort of think of a symlink as having 2 owners: the user that owns the symlink itself, and the user who owns the file pointed to by the symlink. One of those owners might be an attacker, so every time you interact with a file, you have to think "this file might be half-owned by an attacker, and half-owned by a victim".