In any other domain, I would agree with you 100%. But nutrition science really is that bad, in my experience and opinion. With some exceptions (e.g. the need for vitamins to avoid things like scurvy, or the relationship between salt intake and blood pressure), even long-standing nutrition beliefs and practices have been overturned (e.g. consumption of cholesterol, or the discovery of the role of dietary fiber), and some of the newer research is likely to overturn others (e.g. with the role and diversity of gut microbiomes, it's likely other nutrition advice will depend to some extent on your specific microbiome).

The reason for this is fairly simple to see: the methods of science that work so well in other areas of biology are completely impractical in nutrition because of

1. The difficulty of ascertaining and maintaining compliance with a specific diet for a long term study

2. The very long-term effect of some food choices

3. The unknown degree of inter-personal variance in food consumption

4. The expected low effect size of dietary recommendations

5. The huge variety of possible dietary effects

6. The huge amount of possible confounding factors in any population-level study

As you'd expect from this combination, the only effects we really have good science about are those that are relatively fast acting (e.g. salt intake increases BP in less than a day) or have very strong effect sizes (e.g. lack of vitamins or certain amino-acids produces severe diseases). For things like life-long effects, or even effects over multiple years, especially where the correlation is slight, you're left with very unclear science where the unknown possible confounding factors dominate any conclusion.

Edit to add: even today, there is a clear disconnect in nutrition science between people who advocate mostly for relatively simple guidelines and the avoidance of processed foods, usually recommending a preference for vegetables over animal-based products; and the older style of guidelines that you suggest, that say a grilled steak is much worse for you than, say, a stevia-sweetened granola bar you'd buy in a super market.