> For the complete documentation index, see [llms.txt](https://ayushthakur.gitbook.io/ayush-thakur/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ayushthakur.gitbook.io/ayush-thakur/authoring/understanding-the-effectivity-of-ensembles-in-dl.md).
# Understanding the Effectivity of Ensembles in DL
The report explores the ideas presented in [Deep Ensembles: A Loss Landscape Perspective](https://arxiv.org/abs/1912.02757) by Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan.
In the paper, the authors investigate the question - **why do deep ensembles work better than single deep neural networks?**
In their investigation, the authors figure out:
* Different snapshots of the same model (i.e., model trained after 1, 10, 100 epochs) exhibit functional similarity. Hence, their ensemble is **less likely** to explore the different modes of local minima in the optimization space.
* Different solutions of the same model (i.e., trained with different random initializations each time) exhibit functional dissimilarity. Hence, their ensemble is **more likely** to explore the different modes of local minima in the optimization space.
Inspired by their findings, in this report, we present several different insights that are useful for understanding the dynamics of deep neural networks in general.
## :love\_you\_gesture: Read the full article [here](https://app.wandb.ai/authors/loss-landscape/reports/Understanding-the-Effectivity-of-Ensembles-in-Deep-Learning--VmlldzoxODAxNjA).
## 😼 Check out the GitHub repo [here](https://github.com/ayulockin/LossLandscape).
#### This was co-written and implemented with [Sayak Paul](https://sayak.dev).