Linked Questions

4
votes
0answers
186 views

Cherry picking and low p-values [duplicate]

Let's say I run a lot of univariate OLS regression models, say 200,000, with 50 data points, then cherry pick the best one (highest r-square). If my $p$-value for this model is way less than 1/200,000,...
9
votes
5answers
6k views

Is a variable significant in a linear regression model?

I've got a linear regression model with the sample and variable observations and I want to know: Whether a specific variable is significant enough to remain included in the model. Whether another ...
20
votes
5answers
614 views

When can you use data-based criteria to specify a regression model?

I've heard that when many regression model specifications (say, in OLS) are considered as possibilities for a dataset, this causes multiple comparison problems and the p-values and confidence ...
25
votes
2answers
2k views

How to cope with exploratory data analysis and data dredging in small-sample studies?

Exploratory data analysis (EDA) often leads to explore other "tracks" that do not necessarily belong to the initial set of hypotheses. I face such a situation in the case of studies with a limited ...
9
votes
3answers
31k views

Bonferroni correction with Pearson's correlation and linear regression

I am running stats on 5 IVs (5 personality traits, extroversion, agreeableness, conscientiousness, neuroticism, openness) against 3 DVs Attitude to PCT, Attitude to CBT, Attitude to PCT vs CBT. I also ...
10
votes
2answers
8k views

multiple regression and multiple comparisons

Say I fit a multiple regression of p explanatory variables. The t-test will allow me to check if any single one of those is significant ($H_0: \beta_i = 0$). I can do a partial F-test to check if some ...
7
votes
2answers
835 views

Is there a multiple testing problem when performing t-tests for multiple coeffcients in linear regression?

This question comes from a discussion on the recent post by @rvl It's all in the family; but do we include the in-laws too? Here's a common scenario that I've seen many times. A researcher runs a ...
1
vote
1answer
5k views

Improving a linear regression prediction

I'm trying to create a prediction model. I have about 10,000 samples comprising of about 100 predictor variables and the response variable. At the moment I'm using a method which is apparently known ...
4
votes
2answers
2k views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
2
votes
0answers
2k views

Bonferroni correction in generalised linear model

I am running a generalised linear model in R. I have a single response variable and a maximum of 4 possible explanatory variables. I am adding each explanatory variable to the model sequentially, ...
2
votes
1answer
1k views

Data splitting in regression

Edit: The key point I'm attempting to understand is whether during a regression model building exercise, do I need separate datasets to: search for predictors and settle on a functional form ...
9
votes
1answer
442 views

It's all in the family; but do we include the in-laws too?

Suppose I have an experiment with two or more factors. An overall ANOVA is constructed, and then we follow-up with two or more sets of post hoc tests, say multiple comparisons. My question is about ...
3
votes
1answer
54 views

p-values in multiple linear regression depend on number of predictors?

So I am wondering if there usually is any "correction" being made to p-Values (of single predictors) if you add more predictors to the model? Does anybody know how SPSS handles this? I am well aware ...
0
votes
1answer
59 views

Post-hoc corrections for multiple t-tests

I am comparing two groups (young vs. old adults) for differences in 5 brain regions (size) with independent t-tests. I also compare performance on three different aspects of a task with independent t-...
1
vote
0answers
71 views

Correction for several linear mixed effect models

I am searching for correlations between various dependent variables and various factors as well as interactions between them, using linear mixed effects models. I am wondering how to calculate the ...