Yes, fvisibility=hidden is a great addition; combined with LTO and a Clang toolchain, you can also add fsanitize=cfi. The CFI sanitizer adds a 1% perf penalty for a significant exploit mitigation. It complements -fcf-protection=full nicely.
You can also add fsanitize=shadow-stack (ARM) or fsanitize=safe-stack (x86_64) for stronger protection than -fstack-protector-all. This will cause many programs to crash.