Skip to main content
Cross Validated

Return to Question

Became Hot Network Question
Tweeted twitter.com/StackStats/status/1549725833815887875
added 156 characters in body
Source Link
stats_noob
stats_noob
  • 1
  • 6
  • 39
  • 124

Suppose I have the following dataset that contains measurements on different patients (each unique ID is a unique patient) - "x" variables represent covariates and "y" variables represent outcomes:

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant. The calculations required to calculate the beta coefficients for "model 1" do not seem to directly affect the calculations required to calculate the beta coefficients for "model 7" - even though both of these models are using the same dataset. This being said, why should testing the statistical significance of coefficients from "model 1" require somehow "correcting" for "model 2" and vice versa?

Can someone please help me understand as to why in this problem that I have outlined, a "correction factor" is required for individual comparisons and why all individual hypothesis tests for these models need to be compared at a lower value of "alpha" - even though all these individual comparisons seem intendentindependent?

Suppose I have the following dataset that contains measurements on different patients - "x" variables represent covariates and "y" variables represent outcomes:

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant. The calculations required to calculate the beta coefficients for "model 1" do not seem to directly affect the calculations required to calculate the beta coefficients for "model 7" - even though both of these models are using the same dataset.

Can someone please help me understand as to why in this problem that I have outlined, a "correction factor" is required and why all hypothesis tests for these models need to be compared at a lower value of "alpha" - even though all these comparisons seem intendent?

Suppose I have the following dataset that contains measurements on different patients (each unique ID is a unique patient) - "x" variables represent covariates and "y" variables represent outcomes:

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant. The calculations required to calculate the beta coefficients for "model 1" do not seem to directly affect the calculations required to calculate the beta coefficients for "model 7" - even though both of these models are using the same dataset. This being said, why should testing the statistical significance of coefficients from "model 1" require somehow "correcting" for "model 2" and vice versa?

Can someone please help me understand as to why in this problem that I have outlined, a "correction factor" is required for individual comparisons and why all individual hypothesis tests for these models need to be compared at a lower value of "alpha" - even though all these individual comparisons seem independent?

added 227 characters in body
Source Link
stats_noob
stats_noob
  • 1
  • 6
  • 39
  • 124

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant. The calculations required to calculate the beta coefficients for "model 1" do not seem to directly affect the calculations required to calculate the beta coefficients for "model 7" - even though both of these models are using the same dataset.

I can understand if someone was interested in determining if all "x_1" coefficients for all models were JOINTLY statistically significant why a correction factor might be needed - but I am having trouble understand why testing whether " -0.09218" is statistically significant by itself still required a correction factor. Where exactly is the "Family" in this case?

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant.

I can understand if someone was interested in determining if all "x_1" coefficients for all models were JOINTLY statistically significant why a correction factor might be needed - but I am having trouble understand why testing whether " -0.09218" is statistically significant by itself still required a correction factor.

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant. The calculations required to calculate the beta coefficients for "model 1" do not seem to directly affect the calculations required to calculate the beta coefficients for "model 7" - even though both of these models are using the same dataset.

I can understand if someone was interested in determining if all "x_1" coefficients for all models were JOINTLY statistically significant why a correction factor might be needed - but I am having trouble understand why testing whether " -0.09218" is statistically significant by itself still required a correction factor. Where exactly is the "Family" in this case?

Source Link
stats_noob
stats_noob
  • 1
  • 6
  • 39
  • 124

Why is a "Correction" Required in Multiple Hypothesis Testing?

Suppose I have the following dataset that contains measurements on different patients - "x" variables represent covariates and "y" variables represent outcomes:

 id       x_1       x_2       x_3        x_4       x_5       y_1       y_2        y_3       y_4       y_5       y_6       y_7        y_8        y_9      y_10
1  1 137.71008  83.22509  96.77519 139.199703 -31.05143  20.78144 201.37399 251.993953 343.69970 231.62894 278.26058  63.90400 179.571790  -7.370581 135.52294
2  2 240.57995  76.04897 -21.00893   3.407949 161.11235 115.08062 101.07260 200.581716 112.08013 177.05726 215.56689 145.16217 109.613811 112.834290  37.73935
3  3  66.92601 179.04881 -34.18888   3.106041 225.28584  56.88339 220.26244   3.416566 172.49610  54.01245  11.05028  53.05884 100.335158  77.712706 296.31327
4  4 120.87087 202.61092 176.98454 140.890102 142.51842 -64.06773 -23.43913 169.147210 129.27064 -70.55632  22.39471  30.87697 147.580290  43.288333 111.33670
5  5  56.29673  48.33402 119.25437  45.504666  89.87237 198.60063 181.48716  24.956806  55.75533 101.08906  18.19344 163.20589   5.623714  85.519361 148.23663
6  6  60.13320 156.73718 -10.77539 -79.319398 -47.54688 127.28152  49.17664 126.881145  37.45836 142.68339 103.83295 231.97907 186.168837 235.994598 230.42569

I am interested in fitting the following regression models - I believe this is called "Multiple Regression":

model_1 <- lm(y_1 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_2 <- lm(y_2 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_3 <- lm(y_3 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_4 <- lm(y_4 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_5 <- lm(y_5 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_6 <- lm(y_6 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_7 <- lm(y_7 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_8 <- lm(y_8 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_9 <- lm(y_9 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)
model_10 <- lm(y_10 ~ x_1 + x_2  + x_3  + x_4  + x_5 , data = my_data)

Based on the results of these regression models, suppose I want to see that if each "beta" coefficient associated with "x_1" for each model can be considered statistically significant or not.

I have been told that for such problems, a "correction" such as the Bonferroni Correction is often required to choose a more "conservative" value of "alpha" that will be used in these hypothesis tests. But I struggle to understand why this necessary. For example, if I were to inspect the results of these models:

> model_1

Call:
lm(formula = y_1 ~ x_1 + x_2 + x_3 + x_4 + x_5, data = my_data)

Coefficients:
(Intercept)          x_1          x_2          x_3          x_4          x_5  
  131.15352     -0.09218     -0.03841      0.02782     -0.07882     -0.05099  

> model_7

Call:
lm(formula = y_7 ~ x_1 + x_2 + x_3 + x_4 + x_5, data = my_data)

Coefficients:
(Intercept)          x_1          x_2          x_3          x_4          x_5  
  108.07151     -0.06348      0.01031     -0.05734     -0.06381     -0.12770

I can't seem to understand why determining if "-0.09218" is statistically significant - how this somehow is related to whether "-0.06348" is statistically significant.

Can someone please help me understand as to why in this problem that I have outlined, a "correction factor" is required and why all hypothesis tests for these models need to be compared at a lower value of "alpha" - even though all these comparisons seem intendent?

I can understand if someone was interested in determining if all "x_1" coefficients for all models were JOINTLY statistically significant why a correction factor might be needed - but I am having trouble understand why testing whether " -0.09218" is statistically significant by itself still required a correction factor.

Thanks!