undefined | Better HN

0 pointssalty_biscuits7y ago0 comments

I really violently oppose this characterization of ML as "just" curve fitting, as if curve fitting is some simple solved problem. It seems like there is a ignorance about issues relating to model selection, which is an essential part of curve fitting. What complexity of model does the data support? Can you keep a distribution over structures that allows uncertain parts of the model to be interrogated? These are the parts of the fitting equation that allow something like "experiments" to be automatically generated as part of the curve fitting.

0 comments

ahartmetz7y ago

Not the same kind of experiment. An experiment in the scientific sense tweaks the process that generates the data, not the interpretation of the data. There is an inspiration / hypothesis creation step between old data and new experiment.

Main differences: A hypothesis is sorta kinda like your model's coefficients, but more generally applicable. And you have no feedback loop between model coefficients and input data.

So yeah, you are doing very sophisticated curve fitting. It is useful alright, it's just not very much like science.

salty_biscuitsOP7y ago

No, it's the same. It is just about having access to control variables.

ahartmetz7y ago

What Chomsky is saying is that the control variables don't exist until you create them because the most telling things don't happen until you have a specific hypothesis and make them happen to test the hypothesis.

1 more reply

srean7y ago

But what you describe is still curve fitting. I say this in spite of some expertise in ML myself. There are some parts of ML that are not fall in the curve fitting family but they are still a small part, for example Markov logic network, some parts of reinforcement learning.

What you are saying is curve fitting with good predictive ability is not trivial, and that is indeed true.

salty_biscuitsOP7y ago

Markov Logic Networks are still about finding coefficients for a probability distribution over some process. My opinion is that there is only curve fitting. There is data and a minimum complexity model that can reproduce the data with minimum error. So do you really believe that there are physical processes where this approach will fail?

srean7y ago

There is more to Markov Logic than estimating parameters. That's the "unification" part of the Markov logic, the analogue of https://en.wikipedia.org/wiki/Unification_(computer_science)

j / k navigate · click thread line to collapse

0 pointssalty_biscuits7y ago0 comments

0 comments

ahartmetz7y ago

Main differences: A hypothesis is sorta kinda like your model's coefficients, but more generally applicable. And you have no feedback loop between model coefficients and input data.

So yeah, you are doing very sophisticated curve fitting. It is useful alright, it's just not very much like science.

salty_biscuitsOP7y ago

No, it's the same. It is just about having access to control variables.

ahartmetz7y ago

1 more reply

srean7y ago

What you are saying is curve fitting with good predictive ability is not trivial, and that is indeed true.

salty_biscuitsOP7y ago

srean7y ago

There is more to Markov Logic than estimating parameters. That's the "unification" part of the Markov logic, the analogue of https://en.wikipedia.org/wiki/Unification_(computer_science)

j / k navigate · click thread line to collapse