0
$\begingroup$

I was going through the An Introduction to Statistical Learning: With Applications in R book and I am facing a bit of difficulty in understanding the below paragraph.I am including the paragraph itself because I do not want to tamper with the original information.

consider an example in which p = 100 and H0 : β1 = β2 =. . . = βp = 0 is true, so no variable is truly associated with the response. In this situation, about 5% of the p-values associated with each variable will be below 0.05 by chance. In other words, we expect to see approximately five small p-values even in the absence of any true association between the predictors and the response. In fact, we are almost guaranteed that we will observe at least one p-value below 0.05 by chance! Hence, if we use the individual t-statistics and associated pvalues in order to decide whether or not there is any association between the variables and the response, there is a very high chance that we will incorrectly conclude that there is a relationship. However, the F-statistic does not suffer from this problem because it adjusts for the number of predictors. Hence, if H0 is true, there is only a 5% chance that the Fstatistic will result in a p-value below 0.05, regardless of the number of predictors or the number of observations.

I am not able to comprehend the relationship between p values and predictors.I tried searching also but everywhere the explanation is regarding p value and sample size only.

$\endgroup$
0
$\begingroup$

The paragraph adresses the problem of multiple testing, and discusses why we cannot just do $p = 100$ individual hypothesis tests with $H_0: \beta_k = 0$ for each individual predictor at the significance level $\alpha = 0.05$. Even if the null hypothesis is true there is still a chance that the specific $\hat{\beta}_k$ calculated from your sample leads to a rejection of $H_0$. The probability of rejecting $H_0$ even if it is true is equal to your significance level. So, in the example from the paragraph with 100 hypothesis tests and a significance level of 5%, the expected number of rejections is $0.05 \times 100 = 5$, even if all null hypotheses are true (assuming independence). And, as stated in the paragraph, the probability of observing at least one p-value below 0.05 is close to 1. If the multiple testing is not adjusted for, you will falsely conclude that there is evidence in your data at a 5% significance level of relation between predictor and response.

The paragraph then suggest using the $F$-statistic instead, and simultaneously test all $\beta_k$'s.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.