0
$\begingroup$

I'm reading about AdaBoost in the The Elements of Statistical Learning and I don't understand the equation 10.2. Below is an excerpt from the book.

The power of AdaBoost to dramatically increase the performance of even a very weak classifier is illustrated in Figure 10.2. The features $X_1 ,...,X_{10}$ are standard independent Gaussian, and the deterministic target $Y$ is defined by

$ Y= \begin{cases} 1 & \text{if}\,\sum_{j=1}^{10}X_j^2>\chi_{10}^2(0.5), &&&&& (10.2) \\ -1 & \text{otherwise}. \end{cases} $

Here $\chi_{10}^2(0.5)=9.34$ is the median of a chi-squared random variable with 10 degrees of freedom (sum of squares of 10 standard Gaussians). There are 2000 training cases, with approximately 1000 cases in each class, and 10,000 test observations. Here the weak classifier is just a “stump”: a two terminal-node classification tree. Applying this classifier alone to the training dataset yields a very poor test set error rate of 45.8%, compared to 50% for random guessing. However, as boosting iterations proceed the error rate steadily decreases, reaching 5.8% after 400 iterations

So, our threshold ($\chi_{10}^2(0.5)$) divides the dataset into 2 classes of equal size. I have 2 questions with that:

  1. Why do we want to have classes of equal size?
  2. Why is $\chi_{10}^2(0.5)$ the median of a chi-squared distribution with df=10? What does 0.5 stand for?
$\endgroup$
4
  • 1
    $\begingroup$ "0.5" stands for the median--the 50th percentile. $\endgroup$
    – whuber
    Sep 18, 2021 at 15:02
  • 1
    $\begingroup$ $\chi_{10}^2(0.5)$ is potentially ambiguous notation, and you could imagine different authors using it to mean (a) the density of a $\chi^2_{10}$ random variable at $0.5$, in R dchisq(0.5,10); (b) the cumulative probability of a $\chi^2_{10}$ random variable at $0.5$, in R pchisq(0.5,10); (c) the upper cumulative probability of a $\chi^2_{10}$ random variable at $0.5$, in R pchisq(0.5,10,lower.tail=FALSE); (d) the inverse cumulative probability of a $\chi^2_{10}$ random variable at $0.5$, in R qchisq(0.5,10). Here from the context it means (d) $\endgroup$
    – Henry
    Sep 18, 2021 at 15:27
  • $\begingroup$ To be precise (a) is likelihood that a variable takes value of 0.5. Density at the single point is 0. $\endgroup$
    – Jedrek369
    Sep 18, 2021 at 17:50
  • $\begingroup$ The density is not zero (just type dchisq(0.5,10) as suggested by @Henry), the probability is. $\endgroup$ Sep 20, 2021 at 8:26

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.