However, I did my PhD right before the (2nd) deep learning revolution and have never understood how people do research in this new field. Most of the things I did and still do are very well defined and rooted in math, so I mostly know if the solution is good after the fact. I'm also keen on implementing these solutions well and mostly from scratch following similar principles there too.
Nowadays there is a lot of stuff coming out that is possible solely due to deep learning. My issue, however, with performing research in the field of deep learning has always been that I don't understand how to measure success. I might more or less randomly get to a system that works and outperforms the state of the art by a small margin but I would still not know if I found something fundamental or was just lucky. The only way to know seems to be purely empirical. Add to this the energy cost of GPUs and the discrepancy of training capacity between individuals, universities and companies and you will probably get more or less a full picture of my perception of the matter.
I did use various deep learning methods as a black box for particular tasks they seemed good at. However, I never really liked doing it and at this point I'm starting to feel like a dinosaur not being able or willing to adapt to the new reality.
What's your take on this? Anybody went through similar issues? How do I get excited about deep learning? Or is it fine to remain skeptical?
