Let $X$ = {0,1,2,3,4} and $Y$ = {0,1}. A probability distribution $D$ defined on $X\times Y$ such that $D_x$ = Binomial(4, 0.5) and $D_{y\mid x}$ = Bernoulli(0.5).

Given the predictor: $h(x)$ = $0$ if $x$ is even, $1$ if $x$ is odd, find $L_D(h)$, where $L_D(h)$ represents the true error of a prediction rule, or the probability $D({x : h(x) \ne f(x)})$.

So because $D$ is $X\times Y$, I have D defined as ${4\choose k}(\frac{1}{2})(\frac{1}{2})^k(\frac{1}{2})^{4-k}$, and $X\times Y$ has 10 valid points in $[0,4] \times [0,1]$.

After this, I am unsure how to further approach the problem in order to calculate $L_D(h)$.

For reference, I am learning machine learning off of this textbook (we have covered Ch2 and most of Ch3): https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf

Any guidance is appreciated. Thank you.

  • 1
    $\begingroup$ I am confused. Do you mean XxY instead of X*Y? Also what is f? If Dy|x does not depend on x, then X and Y are independent. Are you sure? $\endgroup$ – jarauh Feb 8 at 20:54
  • $\begingroup$ Your formula is not an adequate definition of the probability. An adequate one would be a function of an ordered pair $(x,y)\in\{0,1,2,3,4\}\times\{0,1\},$ but yours is a function of some numeric variable "$k.$" Thus, you ought to start by contemplating what $D$ really is and how to write it down. Another approach is to recognize that the error rate doesn't depend on $h$ at all! $\endgroup$ – whuber Feb 8 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.