Let $X$ = {0,1,2,3,4} and $Y$ = {0,1}. A probability distribution $D$ defined on $X\times Y$ such that $D_x$ = Binomial(4, 0.5) and $D_{y\mid x}$ = Bernoulli(0.5).

Given the predictor: $h(x)$ = $0$ if $x$ is even, $1$ if $x$ is odd, find $L_D(h)$, where $L_D(h)$ represents the true error of a prediction rule, or the probability $D({x : h(x) \ne f(x)})$.

So because $D$ is $X\times Y$, I have D defined as ${4\choose k}(\frac{1}{2})(\frac{1}{2})^k(\frac{1}{2})^{4-k}$, and $X\times Y$ has 10 valid points in $[0,4] \times [0,1]$.

After this, I am unsure how to further approach the problem in order to calculate $L_D(h)$.

For reference, I am learning machine learning off of this textbook (we have covered Ch2 and most of Ch3): https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf

Any guidance is appreciated. Thank you.

  • 1
    $\begingroup$ I am confused. Do you mean XxY instead of X*Y? Also what is f? If Dy|x does not depend on x, then X and Y are independent. Are you sure? $\endgroup$ – jarauh Feb 8 at 20:54
  • $\begingroup$ Your formula is not an adequate definition of the probability. An adequate one would be a function of an ordered pair $(x,y)\in\{0,1,2,3,4\}\times\{0,1\},$ but yours is a function of some numeric variable "$k.$" Thus, you ought to start by contemplating what $D$ really is and how to write it down. Another approach is to recognize that the error rate doesn't depend on $h$ at all! $\endgroup$ – whuber Feb 8 at 21:10

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.