Abstract:
The following topics will be discussed:
Learning by the stochastic gradient Langevin dynamics (SGLD);
Grokking (delayed generalization) is discussed as Brownian motion;
The second law of thermodynamics and Eyring's formula of kinetics in learning;
Energetic (gradient descent) and entropic (generalization in grokking) mechanisms of chemical reaction in learning.