# What does this loss function mean? [duplicate]

I had a interview on deep learning internship job yesterday, and the interviewer showed a loss function formula(below) and asked me 2 questions:

$$J(\theta)=-\mathbb{E}_{x,y\sim\hat{p}_{data}}\log{p_{model}(y|x)}$$

1. How to understand $p(y|x)$ in the formula above?
2. On what task does this loss function used for?

Since I have no idea on this loss function, how to answer these 2 questions?

• Not exactly the same, but I think my answer should be good for you. stats.stackexchange.com/questions/241380/… Dec 26, 2016 at 4:13
• As reproduced the loss function is meaningless since there is no $\theta$ on the right hand side. Dec 26, 2016 at 10:24

1. Since $J(\theta)$ is a function of $\theta$, $p_{model}(y|x)$ should be understood as the likelihood function of $\theta$, where $\theta$ is the model parameters, IMO.
Say we have a neural network that takes in $x$ and outputs $f_\theta(x)$, and we assume a Gaussian noise for $y\sim N(f_\theta(x), \sigma^2)$, then $p_{model}(y|x)=\frac{1}{\sqrt{2\sigma^2\pi}}\exp(-\frac{(y-f_\theta(x))^2}{2\sigma^2})$, then $$J(\theta)=\frac{1}{2\sigma^2}\frac{1}{m}\sum (y-f_\theta(x))^2+constant$$ minimizing $J(\theta)$ in this case equals to minimizing our favorite mean-squared-error.
• Where is $\theta$ in your likelihood? Dec 26, 2016 at 10:25
• A question that is intentionally making no mathematical sense?! In your answer, would you consider $\theta$ as implicitly contained in $p(y|x)$ or anything else? For instance, what does $\theta$ stand for in the Normal example? Dec 26, 2016 at 11:14