In some ML algorithms you don't learn parameters. For example: some clustering algorithms are based on examples, not on parameters.

> Yes, this means ML is "just" statistics

So, a decision tree based on information theory would you call it statistics? Information theory and statistics are not clearly the same.

> I thought this was pretty much an accepted definition.

Machine Learning: A machine that learns (regardless it uses statistics, information theory, function optimization, biological inspiration or whatever)

"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E." - Tom Mitchell