Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.
22
votes
2answers
3k views
What if residuals are normally distributed, but y is not?
I've got a weird question. Assume that you have a small sample where the dependent variable that you're going to analyze with a simple linear model is highly left skewed. Thus you assume that $u$ is ...
26
votes
6answers
7k views
How can a regression be significant yet all predictors be non-significant?
My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant.
All the regression assumptions are met. No multicollinearity ...
34
votes
6answers
32k views
In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?
Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
59
votes
6answers
4k views
Is $R^2$ useful or dangerous?
I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
10
votes
3answers
3k views
What if interaction wipes out my direct effects in regression?
In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.
43
votes
1answer
14k views
Interpretation of R's lm() output
the help pages in R assume I know what those numbers mean. I don't :)
I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. There ...
11
votes
2answers
2k views
Interpretation of log transformed predictor
I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed.
In the case of
...
8
votes
3answers
8k views
Significance of coefficients in linear regression: significant t-test vs non-significant F-statistic [duplicate]
I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations.
R gives me the following p-values from the ...
11
votes
2answers
7k views
What is the difference between doing linear regression on y with x versus x with y?
What is the difference between doing linear regression on y with x versus x with y?
I guess my confusion arises from the fact that the Pearson correlation coefficient of x and y is the same no matter ...
4
votes
2answers
1k views
How can adding a 2nd IV make the 1st IV significant?
I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out.
I have a least squares regression model, with one independent variable and one ...
20
votes
8answers
4k views
Do all interactions terms need their individual terms in regression model?
I am actually reviewing a manuscript where the authors compare 5-6 logit regression models with AIC. However, some of the models have interaction terms without including the individual covariate ...
17
votes
7answers
3k views
What is a complete list of the usual assumptions for linear regression?
What are the usual assumptions for linear regression?
Do they include:
a linear relationship between the independent and dependent variable
independent errors
normal distribution of errors
...
25
votes
6answers
5k views
When is it ok to remove the intercept in lm()?
I am running linear regression models and wondering what the conditions are for removing the intercept term of lm()?
In comparing results from two different lm ...
18
votes
3answers
4k views
Does it make sense to add a quadratic term but not the linear term to a model?
I have a (mixed) model in which one of my predictors should a priori only be quadratically related to the predictor (due to the experimental manipulation). Hence, I would like to add only the ...
21
votes
1answer
3k views
Removal of statistically significant intercept term boosts $R^2$ in linear model
In a simple linear model with a single explanatory variable,
$\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$
I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
7
votes
4answers
763 views
Obtaining a formula for prediction limits in a linear model
Let's take the following example:
set.seed(342)
x1 <- runif(100)
x2 <- runif(100)
y <- x1+x2 + 2*x1*x2 + rnorm(100)
fit <- lm(y~x1*x2)
This creates a ...
2
votes
1answer
1k views
How to test the statistical significance for categorical variable in linear regression?
If in a linear regression I have categorical variable... how do I know the stastical signifance of the categorical variable?
Let's say the factor $X_1$ has 10 levels... there will be 10 different ...
6
votes
1answer
616 views
Interpretation of simple predictions to odds ratios in logistic regression
I'm somewhat new to using logistic regression, and a bit confused by a discrepancy between my interpretations of the following values which I thought would be the same:
exponentiated beta values
...
26
votes
3answers
2k views
F and t statistics in a regression
In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests?
In my model, there are 10 ...
17
votes
5answers
1k views
Efficient online linear regression
I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
3
votes
1answer
419 views
Using and interpreting type I SOS in three-way ANOVA
My research question is from education: To test whether groups (university, department, gender) differ based on their scores in a test, and whether there are any interaction effects, I'll do 3-way ...
5
votes
3answers
2k views
X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A,B, C) other (related) variables, X and two other variables (A,B) are significant ...
27
votes
2answers
17k views
How exactly does one “control for other variables”?
Here is the article that motivated this question: Does impatience make us fat?
I liked this article, and it nicely demonstrates the concept of “controlling for other variables” (IQ, career, income, ...
15
votes
2answers
478 views
In what order should you do linear regression diagnostics?
In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty.
The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
14
votes
1answer
413 views
Logistic regression in R resulted in Hauck Donner phenomenon. Now what?
I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is -∞ to ∞). My data set has almost 24,000 rows. When I run glm in R, I get:
...
10
votes
2answers
2k views
When and how to use standardized explanatory variables in linear regression
I have 2 simple questions about linear regression:
When is it advised to standardize the explanatory variables?
Once estimation is carried out with standardized values, how can one predict with new ...
6
votes
4answers
1k views
Averaging correlation values
Let's say I test how variable Y depends on variable X under different experimental conditions and obtain the following graph:
...
10
votes
3answers
726 views
Effect of switching response and explanatory variable in simple linear regression
Let's say that there exists some "true" relationship between $y$ and $x$ such that $y = ax + b + \epsilon$, where $a$ and $b$ are constants and $\epsilon$ is i.i.d normal noise. When I randomly ...
9
votes
2answers
867 views
Determining best fitting curve fitting function out of linear, exponential, and logarithmic functions
Context:
From a question on Mathematics Stack Exchange (Can I build a program), someone has a set of $x-y$ points, and wants to fit a curve to it, linear, exponential or logarithmic.
The usual ...
0
votes
4answers
367 views
Explain regression to 7 years old [closed]
Please give a clean, simple, explanatory answer that any 7 yr old can understand.
You can also link to a regression guide that is very good and simple. Should be fully explanatory all the way ...
47
votes
6answers
3k views
What skills are required to perform large scale statistical analyses?
Many statistical jobs ask for experience with large scale data. What are the sorts of statistical and computational skills that would be need for working with large data sets. For example, how about ...
31
votes
8answers
2k views
Is there an intuitive explanation why multicollinearity is a problem in linear regression?
The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...
18
votes
6answers
5k views
Is adjusting p-values in a multiple regression for multiple comparisons a good idea?
Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
24
votes
5answers
13k views
Rules of thumb for minimum sample size for multiple regression
Within the context of a research proposal in the social sciences, I was asked the following question:
I have always gone by 100 + m (where m
is the number of predictors) when
determining ...
8
votes
1answer
698 views
Least angle regression keeps the correlations monotonically decreasing and tied?
I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing).
Consider a regression ...
12
votes
1answer
2k views
When to use an offset in a Poisson regression?
Does anybody know why offset in a Poisson regression is used? What do you achieve by this?
12
votes
4answers
1k views
Whether to delete cases that are flagged as outliers by statistical software when performing multiple regression?
I am performing multiple regression analyses and I am not sure whether outliers in my data should be deleted. The data I am concerned about appear as "circles" on the SPSS boxplots, however there are ...
11
votes
8answers
3k views
Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?
I am attempting to run an OLS regression:
DV: Change in weight over a year (initial weight - end weight)
IV: Whether or not you exercise.
However, it seems reasonable that heavier people will lose ...
7
votes
6answers
2k views
Estimating a distribution based on three percentiles
What methods can I use to infer a distribution if I know only three percentiles?
For example, I know that in a certain data set, the fifth percentile is 8,135, the 50th percentile is 11,259, and the ...
7
votes
5answers
1k views
Finding the change point in data from a piecewise linear function
Greetings,
I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help!
I have data conforming to a piecewise linear ...
5
votes
3answers
484 views
How can you handle unstable $\beta$ estimates in linear regression with high multi-collinearity without throwing out variables?
Beta stability in linear regression with high multi-collinearity?
Let's say in a linear regression, the variables $x_1$ and $x_2$ has high multi-collinearity (correlation is around 0.9).
We are ...
4
votes
1answer
384 views
Linear regression prediction interval
If the best linear approximation (using least squares) of my data points is the line $y=mx+b$, how can I calculate the approximation error? If I compute standard deviation of differences between ...
3
votes
1answer
163 views
Interpreting interaction terms in logit regression with categorical variables
I have data from a survey experiment in which respondents were randomly assigned to one of four groups:
...
8
votes
3answers
2k views
Adding coefficients to obtain interaction effects - what to do with SEs?
I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
1
vote
2answers
401 views
Left skewed vs. symmetric distribution observed
This is pretty hard for me to describe, but I'll try to make my problem understandable. So first you have to know that I've done a very simple linear regression so far. Before I estimated the ...
1
vote
0answers
196 views
Can I include year as an independent variable in this nested logistic regression design?
I want to make a nested logistic regression in R with the package mlogit.
I would like to test how producer's decision to enter organisations (14 organisations) or not is affected by different ...
1
vote
2answers
576 views
Interpreting main effect and interaction
I am doing a simple marketing project that has the following types of variables:
X1 - continuous (e.g. income)
X2 - categorical (e.g. gender)
Y - continuous (e.g. number of a product type purchased ...
0
votes
1answer
143 views
Why is there a difference in signs of regression coefficients of same variable in a simple regression and multiple regression on spss
I have checked the relationships of dimensions of independent variable with the dependent variable in a simple linear regression on SPSS, but when I performed multiple linear regression on SPSS of the ...
0
votes
2answers
263 views
How to interpret basic output from a regression analysis?
I have been trying to interpret the results below, but I am finding it difficult. I wonder if someone could help me. All answers highly appreciated.
Number of obs = 30
F( 2, 27) = 19.73
Prob > ...
27
votes
8answers
15k views
How should outliers be dealt with in linear regression analysis?
Often times a statistical analyst is handed a set dataset and asked to fit a model using a technique such as linear regression. Very frequently the dataset is accompanied with a disclaimer similar to ...
