That's the point, see my other comment [1]. User has to know about it to activate manual verification, and by default he just has to trust Signal's CA that his contact is, indeed, the one he is talking to.
[1]:https://news.ycombinator.com/item?id=28081152