What about using JSP? It's somewhat Java and doesn't require Java on the front end(which is the source of most security problem).
On the server side, you have to look for the vulnerability of the web container that you are using.
I do not see how would a CA helps you in your concern. Yes, having certs provides you encryption when your data is travelling, and client side cert may also helps you as a method as authentication if you wish, but all those does not help you in the Java security problem itself.
I'd be interested of a different opinion.