Part of it might be that when the experts try to lay it out for us laypeople, their explanations are sometimes simplified to a point where they clearly make no sense, which creates concern. When an expert says that "it is the case that A because of B" but we already know that B is false or impossible, what are we supposed to believe?
For example, when China changed the definition of a confirmed case, the new definition caused a big spike in the numbers. Later they changed back to the old definition (lower numbers) and the new number of old-definition cases was still higher than the new-definition peak, but with a lower rate of growth. The WHO called this an "encouraging trend", which was very confusing. The trend would look encouraging if you didn't know about the change in definition, but with the change in definition in mind it didn't look encouraging at all. Were they operating on a level beyond that where it does look encouraging again?
It would put many people's minds at rest, I think, if the experts would explain their reasoning better. "It looks encouraging. Now, I'm not talking about the superficial encouraging look of this graph here, which is just an illusion because ..., but if you look deeper still it surprisingly turns out to be encouraging nevertheless because ...".