### Machine learning theory and its relations to statistical physics

96

views

1

Recently, I came across this article:

“New Theory Cracks Open the Black Box of Deep Learning”

in Quanta Magazine.

The reference there, by Tishby

So, I added this post to open up a discussion on the theoretical aspects of machine learning and its relations to statistical physics.

It should be possible to open the black box of neural networks with theoretical physics, esp. statistical physics.

#machineLearning #statisticalPhysics

“New Theory Cracks Open the Black Box of Deep Learning”

in Quanta Magazine.

The reference there, by Tishby

*et al*., is interesting for me since it seems to be using well-known ideas from statistical physics -- at a first naïve glance. The cited paper by the physicists Schwab and Mehta is also eye-catching: They discovered that a certain deep-learning algorithm works, in a particular case, exactly like renormalization. It was a stunning indication that “extracting relevant features in the context of statistical physics and extracting relevant features in the context of deep learning are not just similar words, they are one and the same.”So, I added this post to open up a discussion on the theoretical aspects of machine learning and its relations to statistical physics.

It should be possible to open the black box of neural networks with theoretical physics, esp. statistical physics.

#machineLearning #statisticalPhysics

Please login to add an answer/comment or follow this question.

I find that I can understand the section on RBMs much better than that on RG...

How very nice: "These results also provide a natural interpretation for variational RG entirely in the language of probability theory." - Can we please get this for all of physics?