Uhhh most deep nets for classification or even regression have a regression at the end. You can even use a svm at the end. The power of deep learning is in the feature learning.
Yes, but using an SVM or whatever at the end of a deep network only really works for the same domains that deep networks work well in already - typically audio/visual data and some language datasets.
Most "simple" datasets like insurance prediction from structured data has been solved already via traditional techniques. The difficulty in machine learning was always in the noisy and unstructured datasets like images.
I'm curious what background you have in ML. "The difficulty in machine learning was always in the noisy and unstructured datasets like images" is truly a baffling statement to make.
