> cheaper in the long term

Citation needed :)) There are economies of scale and various optimizations that are just not possible with dedicated machines.

Anecdotal evidence: a company I worked in had a dedicated DC with hundreds/thousands of machines that mostly ran SQL queries on petabytes of data (any query would take ~5-30 minutes). Eye-watering budget and whole teams to maintain the cluster... They switched to GCP/BigQuery, got queries that ran in seconds at a fraction of the budget.

"The cloud" simply means remote servers. Those servers may or may not contain application-specific hardware acceleration.

Nowadays many cell phone application processors also have dedicated hardware to accelerate neural nets, but they will always be limited by thermal constraints.

If you can saturate it, sure. The average person is not going to be running 24/7 AI compute tasks though, so paying for a month of A100 dedi when you only need 10-20 minutes of compute/week is a complete waste.