16
votes
Why does the normality assumption not affect Linear Regression in large samples?
The assumption of the Normality of the error term in a regression that applies Least-Squares estimation methods, is used to make statistical inferences about the coefficients after estimation, it is ...
13
votes
Can I use multiple linear regression with binary output?
You can. It's called a linear probability model.
Should you? There are a couple of problems. Someone can only score 0 or 1. What does it mean to predict they score 0.5? (Maybe it means 50% probability)...
- 18.6k
13
votes
What test is most appropriate if you're interested in an interaction, but have more variables than samples
You have far too few observations and far too many predictors to reach conclusions. Whatever you do, you will be chasing noise, even if you correct for multiple comparisons and regularize, unless ...
- 129k
13
votes
Accepted
Dropping outlier from linear regression model reducing adjusted R^2
Seems to be a situation like in the well known Anscombe quartet, the lower right graph with Y4. Probably your situation is less dramatic, in the sense the vertical bar of 9 points (at the left in that ...
- 1,818
13
votes
Is it possible to calculate the power for a specific beta coefficient in a multiple linear regression?
Yes, this is possible.
Eric Vittinghoff and coauthors include sample size equations at the end of the chapters in their book Regression Methods in Biostatistics.
The power $\gamma$ attained given a ...
- 38.6k
12
votes
Accepted
Model reduction in linear regression by stepwise elimination of predictors with "non-significant" coefficients
This procedure looks like standard backward elimination based on p-values except for the "smallest absolute value" selection, which only makes sense if predictors are standardised. The major ...
- 26.8k
11
votes
Accepted
Why do we use an F distribution instead of just chi-squared when testing multiple hypotheses from regression?
Hint
This numerator is distributed as chi-sq.
Almost right, but not quite. The precise statement is that the numerator scaled by the dispersion parameter $\sigma^2$ of error $\varepsilon$ is chi-...
- 20.6k
11
votes
Accepted
How to analyze a dichotomous outcome with 50% missing data?
Your description implies that you committed the mortal sin of not pre-specifying the final model in the sense that you tried different models in a way not unlike stepwise variable selection does (...
- 96.5k
11
votes
Count predictor and binary outcome
Per your questions...
Is a binary logistic regression the best approach when I have a count predictor and a binary outcome?
Yes. Logistic regression handles any linear equation which requires the ...
- 16.2k
11
votes
Accepted
Assumptions of Linear Regression (homoscedasticity and normality of residuals)
The questions themselves are interesting and nontrivial enough that I believe you may have some basic knowledge about assumption testing already, so I'm not telling you what to do in particular (for ...
- 26.8k
10
votes
Accepted
Can I use multiple linear regression with binary output?
Along with Jeremy's excellent answer, I'll focus on the most relevant bit here:
I searched in the stack exchange site, and people suggest logistic
regression. However, I cannot understand why to use ...
- 16.2k
10
votes
Can you run a t-test with regression betas/coefficients from the output of separate models?
Yes.
For a regression of $y$ onto $x_1$ producing $\widehat{\beta}_1$ as an estimate of the effect on $y$ of a 1-unit increase in $x_1$, and another regression of $y$ onto $x_2$ producing $\widehat{\...
- 30.4k
10
votes
Accepted
Interaction term switching sign of main effect
When the interaction (product term) is in the model, the coefficient for the "main effect" (lower-order term) of grid size (0.00774) gives you the expected slope when the number of ...
- 3,876
10
votes
10
votes
Accepted
Is path analysis equivalent to a series of regressions?
Your specific path model is saturated (just identified). Therefore, you get (very close to) the same results whether you run this model with a series of regressions or as a simultaneous path analytic ...
- 3,876
10
votes
Handling non-significant beta coefficients
Any tips on what we can do with the data to change this?
Why change this? A null result is still a result, and if you purposefully seek to find data to support your hypothesis, that isn't really how ...
- 38.6k
10
votes
Accepted
Missing Coefficients in Linear Regression with Multiple Categorical Variables in R
You need to define a reference level for each separate categorical variable, which will be absorbed into the intercept. (Specifically, R does this automatically, by using the alphabetically first ...
- 129k
10
votes
Is it possible to calculate the power for a specific beta coefficient in a multiple linear regression?
Yes, it's possible. You can use for instance the WebPower package's wp.regression function. It requires the effect size in f2 form though. Standardized beta of 0.3 can be seen as medium effect but on ...
- 2,548
8
votes
Addressing Multicollinearity
Note first that standard linear regression only requires that $x$-variables are not perfectly collinear, i.e., the ${\bf X}$-matrix is of lower rank than the number of variables. As @Alex Teush's ...
- 26.8k
8
votes
Does it make sense to talk of "multicollinearity" in the context of simple linear regression?
If there is only a single independent variable (IV) or predictor, multicollinearity is not a concern/issue and a simple linear regression model with a single IV is analogous to a correlation.
- 1,123
8
votes
Regression model not significant, but the predictor significant
Your question is very similar to this question, except that you had a prior hypothesis about the association between $x$ and $y$. As this answer to that question points out, having too many predictors ...
- 97.9k
8
votes
Accepted
Multicollinearity and control variables dilemma
There is so much wrong with what you quoted!
First, collinearity is not the same as correlation and can involve more than two variables.
Second, there is nothing wrong with any of the equations, and ...
- 125k
8
votes
How to analyze a dichotomous outcome with 50% missing data?
If you want to predict what happens at T2 from data at T1, you could run a three classes model with "dropout", "no dropout", and "T2 missing" as the classes.
Note that ...
- 26.8k
8
votes
Count predictor and binary outcome
Is a binary logistic regression the best approach when I have a count
predictor and a binary outcome?
It is certainly one valid approach, probably the most common one. Is it "best"? That ...
- 125k
7
votes
Accepted
The "detectseparation" package. How to interpret its results?
As it says in the printout, these values indicate existence of the maximum likelihood estimates. 0 means they are finite (i.e. no complete separation), whereas infinite values are for completely ...
- 5,238
7
votes
Accepted
Are these two definitions of the coefficient of determination $R^2$ equal?
A good question. I have to confess that I haven't seen the second expression of $R^2$ before (on the other hand, the better-known result that in simple linear regression, $R^2$ is the squared Pearson ...
- 20.6k
7
votes
Accepted
When does SEM have little to no benefit over multiple regression, and there is a distinction without a difference between two approaches?
Relative to multiple regression (and assuming a single DV and only direct effects like you wrote), I see the main advantages of SEM in the possibility to test the model against the observed data (in ...
- 3,876
7
votes
Does it make sense to talk of "multicollinearity" in the context of simple linear regression?
IMO, it does make sense, to some extent. Although it depends on what you mean by "multicollinearity" and "simple linear regression". Many people have distinct definitions for the ...
- 309
7
votes
Accepted
Variable selection in logistic regression
The problem with your proposed approach is that every predictor in itself may not correlate with the outcome, but interactions between them might. Or you might have a curvilinear relationship between ...
- 129k
7
votes
Accepted
Hierarchy principle: who defined it first?
I don't know about "hierarchical", but this is also called the "marginality" principle, (see for example), which goes back at least to Nelder 1977.
- 1,929
Only top scored, non community-wiki answers of a minimum length are eligible