# Rewriting a Deep Generative Model: An Overview

In the words of the authors,

> "Deep network training is a blind optimization procedure where programmers define objectives but not the solutions that emerge. In this paper, we ask if deep networks can be created in a different way. **Can the rules in a network be directly rewritten?**"

The blind optimization procedure is investigated quantitatively in the paper "[Deep Ensembles: A Loss Landscape Perspective](https://arxiv.org/abs/1912.02757)" by Fort et al. Sayak Paul, and I explored this paper in this [report](https://app.wandb.ai/authors/loss-landscape/reports/Understanding-the-Effectivity-of-Ensembles-in-Deep-Learning--VmlldzoxODAxNjA).

The usual recipe for creating a deep neural network is to train such a model on a massive dataset with a defined objective function. This takes a considerable amount of time and is expensive in most cases. The authors of Rewriting a Deep Generative Model propose a method to *create new deep networks by rewriting the rule of an existing pre-trained network* as shown in figure 1. By doing so, they wish to enable novice users to easily modify and customize a model without the training time and computational cost of large-scale machine learning.

![image.png](https://api.wandb.ai/files/authors/images/projects/119547/7ceac491.png)

**Figure 1**: Rewriting GAN without training to remove the watermark, to add people, and to replace the tower with the tree. ([Source](https://arxiv.org/pdf/2007.15646.pdf))

They do so by setting up a new problem statement: **manipulation of specific rules encoded by a deep generative model.**

If you are unfamiliar with deep generative models, here is my [take on the same](https://app.wandb.ai/ayush-thakur/keras-gan/reports/Towards-Deep-Generative-Modeling-with-W-B--Vmlldzo4MDI4Mw).

#### So why is rewriting deep generative model useful?

Deep generative models such as a GAN can learn rich semantic and physical rules about a target distribution(faces, etc.). However, it usually takes weeks to achieve state of the art GAN on any dataset. If the target distribution changes by some amount, retraining a GAN would be a waste of resources. However, what if we directly change some trained GAN rules to reflect the target distribution change? Thus by rewriting a GAN:

* We can build a new model without retraining, which is a more involved task.
* From the perspective of demystifying deep neural nets, the approach to edit a model gives new insight about the model and how semantic features are captured.
* It can also provide some insight into the generalization of deep models to unseen scenarios.
* Unlike conventional image editing tools where the desired change is applied on a single image, by editing a GAN, one can apply the edit on every image generated.
* Using this tool, one can build new generative models without domain expertise, training time, and computational expense.

{% embed url="<http://www.youtube.com/watch?v=iBpdJ2CopIE>" %}

## ​🤠 Read the full report [here](https://wandb.ai/authors/rewrite-gan/reports/Rewriting-a-Deep-Generative-Model-An-Overview--VmlldzoyMzgyNTU). <a href="#read-the-full-article-here" id="read-the-full-article-here"></a>