Machine learning theory and its relations to statistical physics
6 months ago by
“New Theory Cracks Open the Black Box of Deep Learning”
in Quanta Magazine.
The reference there, by Tishby et al., is interesting for me since it seems to be using well-known ideas from statistical physics -- at a first naïve glance. The cited paper by the physicists Schwab and Mehta is also eye-catching: They discovered that a certain deep-learning algorithm works, in a particular case, exactly like renormalization. It was a stunning indication that “extracting relevant features in the context of statistical physics and extracting relevant features in the context of deep learning are not just similar words, they are one and the same.”
So, I added this post to open up a discussion on the theoretical aspects of machine learning and its relations to statistical physics.
It should be possible to open the black box of neural networks with theoretical physics, esp. statistical physics.
Please login to add an answer/comment or follow this question.