Only if it is shared annotations is it a problem.
It need not be shared , think more like a public notion/ share point document with comments visible . I.e experts(users) can create their own individual annotated versions and share with others .
As long as there is no single version of the annotations , moderation is not needed