Frankly, I'm not sure if LLM would be the best replacement. Given they need to make decisions, I'd say ensemble methods could work better, at least in scenarios where you can actually perform reliable simulations (e.g. production, but not necessarily marketing). LLMs tend to averages whereas what shareholders want is to optimize for X, where X is usually, but not necessarily, net profit.

(It would be a fun experiment to create such an optimization startup and advertise it to boards as "AI-CEO-as-a-service" and watch the faces of these CEOs pushing for AI in workplace. Maybe they would start having a more nuanced view.)