Skip to main content
deleted 2 characters in body
Source Link
Paze
  • 2.3k
  • 1
  • 15
  • 33

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

EDIT: After reading your edit.

This may be my main point: does significance at the regression coefficient level mean anything in this case?

The coefficient tells you both the direction and size of the impact your covariates have on the independentdependent variable, with respect to the model as a whole. If the p-value of a covariate is significant, but the model contains enough individual covariates with no pre-determined hypothesis, that you suspect the unadjusted p-values may be erroneous, the coefficient and the CI may be inflated, even if truly significant. Another important part with respect to coefficients is how well you have controlled for confounders as these can dramatically change the coefficients in your model. Related literature on this topic.

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

EDIT: After reading your edit.

This may be my main point: does significance at the regression coefficient level mean anything in this case?

The coefficient tells you both the direction and size of the impact your covariates have on the independent variable, with respect to the model as a whole. If the p-value of a covariate is significant, but the model contains enough individual covariates with no pre-determined hypothesis, that you suspect the unadjusted p-values may be erroneous, the coefficient and the CI may be inflated, even if truly significant. Related literature on this topic.

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

EDIT: After reading your edit.

This may be my main point: does significance at the regression coefficient level mean anything in this case?

The coefficient tells you both the direction and size of the impact your covariates have on the dependent variable, with respect to the model as a whole. If the p-value of a covariate is significant, but the model contains enough individual covariates with no pre-determined hypothesis, that you suspect the unadjusted p-values may be erroneous, the coefficient and the CI may be inflated, even if truly significant. Another important part with respect to coefficients is how well you have controlled for confounders as these can dramatically change the coefficients in your model. Related literature on this topic.

added 656 characters in body
Source Link
Paze
  • 2.3k
  • 1
  • 15
  • 33

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

EDIT: After reading your edit.

This may be my main point: does significance at the regression coefficient level mean anything in this case?

The coefficient tells you both the direction and size of the impact your covariates have on the independent variable, with respect to the model as a whole. If the p-value of a covariate is significant, but the model contains enough individual covariates with no pre-determined hypothesis, that you suspect the unadjusted p-values may be erroneous, the coefficient and the CI may be inflated, even if truly significant. Related literature on this topic.

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.

EDIT: After reading your edit.

This may be my main point: does significance at the regression coefficient level mean anything in this case?

The coefficient tells you both the direction and size of the impact your covariates have on the independent variable, with respect to the model as a whole. If the p-value of a covariate is significant, but the model contains enough individual covariates with no pre-determined hypothesis, that you suspect the unadjusted p-values may be erroneous, the coefficient and the CI may be inflated, even if truly significant. Related literature on this topic.

Source Link
Paze
  • 2.3k
  • 1
  • 15
  • 33

If I'm reading your question correctly you are simply asking about multiple comparison in statistics which is a well known phenomena. To remedy this, you can correct your significance using something like a Bonferroni-correction, although many types of correction methods exist.

Note that you also need to consider the implication of your analysis. Is it something that will dictate how your company or medical department operates? Then you should probably take it more seriously than if your study is simply to pave way and direct future research. Multiple comparison and how we address it, is in essence just a dance between type-1 and type-2 errors.