# Tagged Questions

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

4k views

### What if residuals are normally distributed, but y is not?

I've got a weird question. Assume that you have a small sample where the dependent variable that you're going to analyze with a simple linear model is highly left skewed. Thus you assume that $u$ is ...
45k views

### In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
6k views

### Is $R^2$ useful or dangerous?

I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
10k views

### How can a regression be significant yet all predictors be non-significant?

My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant. All the regression assumptions are met. No multicollinearity ...
13k views

### What is the difference between linear regression on y with x and x with y?

The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
20k views

### Interpretation of R's lm() output

the help pages in R assume I know what those numbers mean. I don't :) I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. There ...
4k views

### What if interaction wipes out my direct effects in regression?

In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.
4k views

### Interpretation of log transformed predictor

I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed. In the case of ...
7k views

### Does it make sense to add a quadratic term but not the linear term to a model?

I have a (mixed) model in which one of my predictors should a priori only be quadratically related to the predictor (due to the experimental manipulation). Hence, I would like to add only the ...
11k views

### Significance of coefficients in linear regression: significant t-test vs non-significant F-statistic [duplicate]

I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations. R gives me the following p-values from the ...
2k views

### How can adding a 2nd IV make the 1st IV significant?

I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out. I have a least squares regression model, with one independent variable and one ...
1k views

### Interpretation of simple predictions to odds ratios in logistic regression

I'm somewhat new to using logistic regression, and a bit confused by a discrepancy between my interpretations of the following values which I thought would be the same: exponentiated beta values ...
7k views

### When is it ok to remove the intercept in lm()?

I am running linear regression models and wondering what the conditions are for removing the intercept term of lm()? In comparing results from two different lm ...
992 views

### Obtaining a formula for prediction limits in a linear model

Let's take the following example: set.seed(342) x1 <- runif(100) x2 <- runif(100) y <- x1+x2 + 2*x1*x2 + rnorm(100) fit <- lm(y~x1*x2) This creates a ...
6k views

### Do all interactions terms need their individual terms in regression model?

I am actually reviewing a manuscript where the authors compare 5-6 logit regression models with AIC. However, some of the models have interaction terms without including the individual covariate ...
4k views

### What is a complete list of the usual assumptions for linear regression?

What are the usual assumptions for linear regression? Do they include: a linear relationship between the independent and dependent variable independent errors normal distribution of errors ...
2k views

### How to test the statistical significance for categorical variable in linear regression?

If in a linear regression I have categorical variable... how do I know the stastical signifance of the categorical variable? Let's say the factor $X_1$ has 10 levels... there will be 10 different ...
4k views

### Removal of statistically significant intercept term boosts $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
2k views

### Whether to delete cases that are flagged as outliers by statistical software when performing multiple regression?

I am performing multiple regression analyses and I am not sure whether outliers in my data should be deleted. The data I am concerned about appear as "circles" on the SPSS boxplots, however there are ...
4k views

### X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?

X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant ...
469 views

### Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
768 views

### Regression based for example on days of week

I need a little help to move in the right direction. It's a long time since I studied any stats and the jargon seems to have changed. Imagine that I have a set of car-related data such as Journey ...
2k views

### Efficient online linear regression

I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
689 views

### Logistic regression in R resulted in Hauck Donner phenomenon. Now what?

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is -∞ to ∞). My data set has almost 24,000 rows. When I run glm in R, I get: ...
750 views

### Testing for moderation with continuous vs. categorical moderators

I am testing an interaction effect where X and Y are continuous variable and M (Moderator) is a categorical variable (effects coding +1, -1). I have no clue about how to do a post-hoc probing of ...
22k views

### Rules of thumb for minimum sample size for multiple regression

Within the context of a research proposal in the social sciences, I was asked the following question: I have always gone by 100 + m (where m is the number of predictors) when determining ...
2k views

### F and t statistics in a regression

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 ...
2k views

### Box-Cox like transformation for independent variables?

Is there a Box-Cox like transformation for independent variables? That is, a transformation that optimizes the $x$ variable so that the y~f(x) will make a more ...
3k views

### Averaging correlation values

Let's say I test how variable Y depends on variable X under different experimental conditions and obtain the following graph: ...
518 views

### Using and interpreting type I SOS in three-way ANOVA

My research question is from education: To test whether groups (university, department, gender) differ based on their scores in a test, and whether there are any interaction effects, I'll do 3-way ...
1k views

### Determining best fitting curve fitting function out of linear, exponential, and logarithmic functions

Context: From a question on Mathematics Stack Exchange (Can I build a program), someone has a set of $x-y$ points, and wants to fit a curve to it, linear, exponential or logarithmic. The usual ...
1k views

### Wald test in regression (OLS and GLMs): t- vs. z-distribution

I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215):  ...
4k views

### What skills are required to perform large scale statistical analyses?

Many statistical jobs ask for experience with large scale data. What are the sorts of statistical and computational skills that would be need for working with large data sets. For example, how about ...
25k views

### What's the difference between correlation and simple linear regression?

In particular, I am referring to the Pearson product-moment correlation coefficient.
25k views

### How exactly does one “control for other variables”?

Here is the article that motivated this question: Does impatience make us fat? I liked this article, and it nicely demonstrates the concept of “controlling for other variables” (IQ, career, income, ...
549 views

### In what order should you do linear regression diagnostics?

In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty. The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
7k views

### Coefficient of Determination ($r^2$): I have never fully grasped the interpretation

I want to fully grasp the notion of $r^2$ describing the amount of variation between variables. Every web explanation is a bit mechanical and obtuse. I want to "get" the concept, not just mechanically ...
4k views

### When and how to use standardized explanatory variables in linear regression

I have 2 simple questions about linear regression: When is it advised to standardize the explanatory variables? Once estimation is carried out with standardized values, how can one predict with new ...
1k views

### Effect of switching response and explanatory variable in simple linear regression

Let's say that there exists some "true" relationship between $y$ and $x$ such that $y = ax + b + \epsilon$, where $a$ and $b$ are constants and $\epsilon$ is i.i.d normal noise. When I randomly ...
1k views

### How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
863 views

### How to find a good fit for semi-sinusoidal model in R?

I want to assume that the sea surface temperature of the Baltic Sea is the same year after year, and then describe that with a function / linear model. The idea I had was to just input year as a ...
27k views

### Regression with multiple dependent variables?

Is it possible to have a (multiple) regression equation with two or more dependent variables? Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it ...
525 views

### Linear regression prediction interval

If the best linear approximation (using least squares) of my data points is the line $y=mx+b$, how can I calculate the approximation error? If I compute standard deviation of differences between ...
5k views

### PCA and proportion of variance explained

in general, what is meant when it says that the fraction X of the variance in an analysis like PCA is explained by the first principal component? can someone explain this intuitively but also give a ...
2k views

### Adding coefficients to obtain interaction effects - what to do with SEs?

I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
475 views

### Explain regression to 7 years old [closed]

Please give a clean, simple, explanatory answer that any 7 yr old can understand. You can also link to a regression guide that is very good and simple. Should be fully explanatory all the way ...
5k views

### Why is ANOVA taught / used as if it is a different research methodology compared to linear regression?

ANOVA is equivalent to linear regression with the use of suitable dummy variables. The conclusions remain the same irrespective of whether you use ANOVA or linear regression. In light of their ...
20k views

### How should outliers be dealt with in linear regression analysis?

Often times a statistical analyst is handed a set dataset and asked to fit a model using a technique such as linear regression. Very frequently the dataset is accompanied with a disclaimer similar to ...