# Calculating Expectation Value of A Loss Function

I have a loss function given by:

$$\mathcal{L}(x, y) = \frac{\sum_{i=0}\sqrt{x_{i}y_{i}}(y_{i} -g(x_{i}))^{2}}{\sum_{i=0}\sqrt{x_{i}y_{i}}}$$

where $$g(x)$$ is some function to be minimized, $$y \in [0,1]$$ and $$x \in [0, \infty]$$

Apparently, this loss function overestimates the value of $$g(x)$$ when values of $$x$$ are large and underestimates the the value of $$g(x)$$ when it is small, but this was not qualified further in my source.

I would like to try and see if this is the case, analytically. My limited understanding would say that by finding the value of $$g(x)$$ that minimizes the expectation value and observe how it related to the value of $$x$$, I would be able to argue this point.

I have attempted, following the example here, to derive the value of $$g(x)$$ that minimizes the expectation value for the loss, $$\mathbf{E}(\mathcal{L})$$. I am very rusty on my probabilities, so I would appreciate any advice if I have messed up. Particularly, I am not sure whether I have treated $$x$$ correctly throughout.

My attempt goes as follows:

$$\mathbf{E}(\mathcal{L}) = \frac{\int f(x|y) \sqrt{xy}(y - g(x)))^{2} \ dy}{\int f(x|y) \sqrt{xy} \ dy}$$

Starting with the denominator, we integrate over $$y$$ to get:

$$\int f(y|x) \sqrt{xy} \ dy = \sqrt{x}\mathbf{E}(\sqrt{y}|x)$$

Then, we look at the numerator and multiply it out:

$$\int f(y|x) \sqrt{xy}(y - g(x))^{2}.dy = \int f(y|x) \sqrt{xy}(y^{2} - 2yg(x) + g(x)^{2}) \ dy$$

We then group the terms:

$$\int f(y|x) \sqrt{xy}y^{2} \ dy - \int f(y|x) 2y\sqrt{y}g(x) \ dy + \int f(y|x)\sqrt{y}g(x)^{2} \ dy$$

We can then integrate over $$y$$ to get:

$$\sqrt{x}(\mathbf{E}(\sqrt{y}y^{2} | x) - 2g(x)\mathbf{E}(\sqrt{y}y | x)) + g(x)^{2}\mathbf{E}(\sqrt{y} | x))$$

We divide the numerator by the denominator to get:

$$\mathbf{E}(\mathcal{L}) = \frac{(\mathbf{E}(\sqrt{y}y^{2} | x) - 2g(x)\mathbf{E}(\sqrt{y}y | x)) + g(x)^{2}\mathbf{E}(\sqrt{y} | x))}{\mathbf{E}(\sqrt{y}|x)}$$

We take the derivative with respect to $$g(x)$$:

$$\frac{d\mathbf{E}(\mathcal{L})}{dg(x)} = \frac{-2\mathbf{E}(\sqrt{y}y|x))}{\mathbf{E}(\sqrt{y}|x)} + 2g(x) = 0$$

Thus we conclude that the value of $$g(x)$$ that leads to the optimal expectation value, $$\min \mathbf{E}(\mathcal{L})$$, of this loss function is given by:

$$g(x) = \frac{\mathbf{E}(\sqrt{y}y|x)}{\mathbf{E}(\sqrt{y}|x)}$$

Assuming this derivation is correct, I do not see how I can infer the reasoning of the source to this result.

Have I achieved the result correctly? If not, where have I messed up?

Is the hypothesis given in the source a valid one, given this result?