Ayush Thakur
  • About Me
  • Authoring
    • DeepFaceDrawing: An Overview
    • Rewriting a Deep Generative Model: An Overview
    • Unsupervised Visual Representation Learning with SwAV
    • In-Domain GAN Inversion for Real Image Editing
    • Metric Learning for Image Search
    • Object Localization with Keras and W&B
    • Image Segmentation Using Keras and W&B
    • Understanding the Effectivity of Ensembles in DL
    • Modern Data Augmentation Techniques for CV
    • Adversarial Latent Autoencoders
    • Towards Deep Generative Modeling with W&B
    • Interpretability in Deep Learning - CAM and GradCAM
    • Introduction to image inpainting with deep learning
    • Simple Ways to Tackle Class Imbalance
    • Debugging Neural Networks with PyTorch
    • Generating Digital Painting Lighting Effects
    • Multi Task Learning with W&B
    • Translate American Sign Language Using CNN
    • Converting FC Layers to Conv Layers
  • Projects
    • Smart Traffic Management Using Reinforcement Learning
    • Sign Language Translator
Powered by GitBook
On this page
  • Read the full article here.
  • ​😼 Check out the GitHub repo here.

Was this helpful?

  1. Authoring

Understanding the Effectivity of Ensembles in DL

Dissecting ensembles, one at a time.

PreviousImage Segmentation Using Keras and W&BNextModern Data Augmentation Techniques for CV

Last updated 4 years ago

Was this helpful?

The report explores the ideas presented in by Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan.

In the paper, the authors investigate the question - why do deep ensembles work better than single deep neural networks?

In their investigation, the authors figure out:

  • Different snapshots of the same model (i.e., model trained after 1, 10, 100 epochs) exhibit functional similarity. Hence, their ensemble is less likely to explore the different modes of local minima in the optimization space.

  • Different solutions of the same model (i.e., trained with different random initializations each time) exhibit functional dissimilarity. Hence, their ensemble is more likely to explore the different modes of local minima in the optimization space.

Inspired by their findings, in this report, we present several different insights that are useful for understanding the dynamics of deep neural networks in general.

Read the full article .

​😼 Check out the GitHub repo .

This was co-written and implemented with .

🤟
here
here
Sayak Paul
Deep Ensembles: A Loss Landscape Perspective
Trajectories taken by same model trained with different inits.