# VAEs: Basic Math example for pass through VAE

As great as many papers and videos are at teaching Neural Net concepts, I see a surprising lack of basic numerical examples explaining these concepts. These examples would let me make sure I know what's going on under the hood and give me intuition for how these models behave when interpreting results. Admittedly, I haven't looked too much into Coursera, Udemy, etc. But I prefer good ol'-fashioned pen and paper. Any suggestions?

tutorial

Siraj

### An example of what I'm looking for:

We want to explain addition to babies. We show them examples: 1+1 = 2, 2+1 = 3, and 1 apple next to 2 apples is 3 apples. Without VAEs broken down so basically, it's hard for me to really understand how things are working

## 1 Answer

I'm not sure if explaining VAE on concrete examples is a good idea (can you do it with a deep net? the point of these models is to stack multiple layers so they would be able to approximate complex functions).

That being said, there are simpler models that work in similar way that use variational inference - for example EM algorithm. Some concrete examples can be found in Variational Inference tutorial that was written by David Blei.

You can go from this to VAE just like you'd go from linear regression to Multilayer Perceptron regressor.

• in EM there is a closed-form solution for what to do in next iteration (like closed-form solution for least squares linear regression)
• note that your update based on ELBO (a proxy for likelihood) can't be done in one step (similarly you can't get closed-form solution for MLP reg) and you need to fall back to gradient descent or similar algorithms.