# What do "Expectation" mean in cost function?

I want to know whether my understanding for expectation of loss function is correct or not.

(x,y) means sample data-label distribution from true distribution. And function f which is respect to weight w is a mapping function from data x to label dimension. And l means loss function that takes label and predicted label. so applying loss function to all data points we have, we can create distribution of loss values for all data points. using this distribution, we can calculate expectation value for all data points(say averaged(?) loss value for all data points?). is my understanding about expectation correct or not?

• I think you have it right. We can compute a loss value for each observation in our data set (if the loss is squared loss this is just the square of the residual). We then want to minimize the average loss across our sample as opposed to any one individual loss value. Mar 3, 2022 at 14:35

The subscript notation is pretty common in machine learning literature, you can find some examples if it being discusses in the following threads: Expected value notation in GAN loss, Notation: What does the tilde below of the expectation mean?, Notation for expected value, Notation: What does the tilde below of the expectation mean?, What is the meaning of superscript in $p_{\theta}(x)$ and ${\mathbb E}_{\theta}\left[S(\theta)\right]$?.