diff --git a/EnergyBasedModel/README.md b/EnergyBasedModel/README.md index e86dfff..065d157 100644 --- a/EnergyBasedModel/README.md +++ b/EnergyBasedModel/README.md @@ -60,6 +60,22 @@ In the example above, blue dots are data points. As you could see, the data are ![Inference with latent variables](./imgs/inference_with_latent_var.png) +## Energy-Based Models vs Probabilistic Models + +- Probabilistic model is a special case of energy-based model (Energies are like unnormalised negative log probabilities) + +- Why use EBM instead of probabilistic models? + + 1) EBM gives more flexibility in the choice of the sciring function + + 2) More flexibility in the choice of objective function for learning + +- From energy to probability: Gibbs Boltzmann distribution (Beta is a positive constant) + +![Gibbs Boltzmann distribution](./imgs/gibbs_boltzmann.png) + +![Marginalizing over the latent variable](./imgs/marginalizing_over_latent_variable.png) + ## References [1] Yann LeCun [Lecture: Energy based models and self-supervised learning](https://www.youtube.com/watch?v=tVwV14YkbYs&list=PLLHTzKZzVU9eaEyErdV26ikyolxOsz6mq&index=12) diff --git a/EnergyBasedModel/imgs/gibbs_boltzmann.png b/EnergyBasedModel/imgs/gibbs_boltzmann.png new file mode 100644 index 0000000..3431183 Binary files /dev/null and b/EnergyBasedModel/imgs/gibbs_boltzmann.png differ diff --git a/EnergyBasedModel/imgs/marginalizing_over_latent_variable.png b/EnergyBasedModel/imgs/marginalizing_over_latent_variable.png new file mode 100644 index 0000000..4efff49 Binary files /dev/null and b/EnergyBasedModel/imgs/marginalizing_over_latent_variable.png differ