1
$\begingroup$

I have a loss function given by:

$$ \mathcal{L}(x, y) = \frac{\sum_{i=0}\sqrt{x_{i}y_{i}}(y_{i} -g(x_{i}))^{2}}{\sum_{i=0}\sqrt{x_{i}y_{i}}} $$

where $g(x)$ is some function to be minimized, $y \in [0,1]$ and $x \in [0, \infty]$

Apparently, this loss function overestimates the value of $g(x)$ when values of $x$ are large and underestimates the the value of $g(x)$ when it is small, but this was not qualified further in my source.

I would like to try and see if this is the case, analytically. My limited understanding would say that by finding the value of $g(x)$ that minimizes the expectation value and observe how it related to the value of $x$, I would be able to argue this point.

I have attempted, following the example here, to derive the value of $g(x)$ that minimizes the expectation value for the loss, $\mathbf{E}(\mathcal{L})$. I am very rusty on my probabilities, so I would appreciate any advice if I have messed up. Particularly, I am not sure whether I have treated $x$ correctly throughout.

My attempt goes as follows:

$$\mathbf{E}(\mathcal{L}) = \frac{\int f(x|y) \sqrt{xy}(y - g(x)))^{2} \ dy}{\int f(x|y) \sqrt{xy} \ dy} $$

Starting with the denominator, we integrate over $y$ to get:

$$\int f(y|x) \sqrt{xy} \ dy = \sqrt{x}\mathbf{E}(\sqrt{y}|x)$$

Then, we look at the numerator and multiply it out:

$$\int f(y|x) \sqrt{xy}(y - g(x))^{2}.dy = \int f(y|x) \sqrt{xy}(y^{2} - 2yg(x) + g(x)^{2}) \ dy$$

We then group the terms:

$$\int f(y|x) \sqrt{xy}y^{2} \ dy - \int f(y|x) 2y\sqrt{y}g(x) \ dy + \int f(y|x)\sqrt{y}g(x)^{2} \ dy$$

We can then integrate over $y$ to get:

$$\sqrt{x}(\mathbf{E}(\sqrt{y}y^{2} | x) - 2g(x)\mathbf{E}(\sqrt{y}y | x)) + g(x)^{2}\mathbf{E}(\sqrt{y} | x))$$

We divide the numerator by the denominator to get:

$$\mathbf{E}(\mathcal{L}) = \frac{(\mathbf{E}(\sqrt{y}y^{2} | x) - 2g(x)\mathbf{E}(\sqrt{y}y | x)) + g(x)^{2}\mathbf{E}(\sqrt{y} | x))}{\mathbf{E}(\sqrt{y}|x)}$$

We take the derivative with respect to $g(x)$:

$$\frac{d\mathbf{E}(\mathcal{L})}{dg(x)} = \frac{-2\mathbf{E}(\sqrt{y}y|x))}{\mathbf{E}(\sqrt{y}|x)} + 2g(x) = 0$$

Thus we conclude that the value of $g(x)$ that leads to the optimal expectation value, $\min \mathbf{E}(\mathcal{L})$, of this loss function is given by:

$$g(x) = \frac{\mathbf{E}(\sqrt{y}y|x)}{\mathbf{E}(\sqrt{y}|x)}$$

Assuming this derivation is correct, I do not see how I can infer the reasoning of the source to this result.

Have I achieved the result correctly? If not, where have I messed up?

Is the hypothesis given in the source a valid one, given this result?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.