# Tagged Questions

This tag is a signal that the question focuses on a problem particular to multivariate analysis, such as multiple correlations or interactions.

18k views

### When should you center your data & when should you standardize?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
10k views

### How can a regression be significant yet all predictors be non-significant?

My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant. All the regression assumptions are met. No multicollinearity ...
716 views

### How well can multiple regression really “control for” covariates?

We’re all familiar with observational studies that attempt to establish a causal link between a nonrandomized predictor X and an outcome by including every imaginable potential confounder in a ...
7k views

### Is adjusting p-values in a multiple regression for multiple comparisons a good idea?

Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
5k views

### Multivariate multiple regression in R

I have 2 dependent variables (DVs) each of whose score may be influenced by the set of 7 independent variables (IVs). DVs are continuous, while the set of IVs consists of a mix of continuous and ...
549 views

### In what order should you do linear regression diagnostics?

In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty. The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
5k views

### How to present results of a Lasso using glmnet?

I would like to find predictors for a continuous dependent variable out of a set of 30 independent variables. I am using Lasso regression as implemented in the glmnet package in R. Here is some dummy ...
412 views

### Bayesian variable selection — does it really work?

I thought I might toy with some Bayesian variable selection, following a nice blog post and the linked papers therein. I wrote a program in rjags (where I am quite a rookie) and fetched price data ...
11k views

### Significance of coefficients in linear regression: significant t-test vs non-significant F-statistic [duplicate]

I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations. R gives me the following p-values from the ...
3k views

### Either quadratic or interaction term is significant in isolation, but neither are together

As part of an assignment, I had to fit a model with two predictor variables. I then had to draw a plot of the models' residuals against one of the included predictors and make changes based on that. ...
12k views

### Acceptable r-square value for multiple linear regression model

I'm currently working on my thesis, more specifically I'm analyzing some data collected from researchers about the project's they're working on. In the end, I have performed a multiple linear ...
4k views

### How to deal with multicollinearity when performing variable selection?

I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
2k views

### Multicollinearity when individual regressions are significant, but VIFs are low

I have 6 variables ($x_{1}...x_{6}$) that I am using to predict $y$. When performing my data analysis, I first tried a multiple linear regression. From this, only two variables were significant. ...
342 views

### Suppression effect in regression: definition and visual explanation/depiction

What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
1k views

### Does the order of explanatory variables matter when calculating their regression coefficients?

At first I thought the order didn’t matter, but then I read about the gram-schmidt orthogonalization process for calculating multiple regression coefficients, and now I’m having second thoughts. ...
446 views

### If I repeat every sample observation in a linear regression model and rerun the regression how would the result be affected?

Say I have N observations, possibly multiple factors and I repeat each observation twice (or M times) how would a regression on this new set of size NM compare to a regression on just the original ...
721 views

### Multiple imputation and model selection

Multiple imputation is fairly straightforward when you have an a priori linear model that you want to estimate. However, things seem to be a bit trickier when you actually want to do some model ...
5k views

### VIF, condition Index and eigenvalues

I am currently assessing multicollinearity in my datasets. What threshold values of VIF and condition index below/above suggest a problem? VIF: I have heard that VIF $\geq 10$ is a problem. After ...
245 views

### Is there a generalization of Pillai trace and the Hotelling-Lawley trace?

In the setting of multivariate multiple regression (vector regressor and regressand), the four major tests for the general hypothesis (Wilk's Lambda, Pillai-Bartlett, Hotelling-Lawley, and Roy's ...
1k views

### Transforming variables for multiple regression in R

I am trying to perform a multiple regression in R. However, my dependent variable has the following plot: Here is a scatterplot matrix with all my variables ...
250 views

### Possible range of $R^2$

Suppose are three time series, $X_1$, $X_2$ and $Y$ Running ordinary linear regression on $Y$ ~ $X_1$ ($Y = b X_1 + b_0 + \epsilon$ ), we get $R^2 = U$. The ordinary linear regression $Y$ ~ $X_2$ get ...
1k views

### How can I use the value of $R^2$ to test the linearity assumption in multiple regression analysis?

The below graphs are residual scatter plots of a regression test for which "normality", "homoscedasticity" and "independence" assumptions have already been met for sure! For testing the "linearity" ...
146 views

### Multicollinearity and Splines :: Is there a problem?

When using natural (i.e. restricted) cubic splines, the basis functions created are highly collinear, and when used in a regression seem to produce very high VIF (variance inflation factor) ...
208 views

### Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other ...
2k views

### How can adding a 2nd IV make the 1st IV significant?

I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out. I have a least squares regression model, with one independent variable and one ...
2k views

### Positive correlation and negative regressor coefficient sign

Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
3k views

### Interpreting coefficient in a linear regression model with categorical variables

I will give my examples with R calls. First a simple example of a linear regression with a dependent variable 'lifespan', and two continuous explanatory variables. ...
523 views

### Help with SEM modeling (OpenMx, polycor)

I have lots of problem with one data set to which I am trying to apply SEM. We suppose the existence of 5 latent factors A, B, C, D, E, with indicators resp. A1 to A5 (ordered factors), B1 to B3 ...
482 views

### Should I run separate regressions for every community, or can community simply be a controlling variable in an aggregated model?

I am running an OLS model with a continuous asset index variable as the DV. My data is aggregated from three similar communities in close geographic proximity to one another. Despite this, I thought ...
358 views

### Coefficients paths – comparison of ridge, lasso and elastic net regression

I would like to compare models selected with ridge, lasso and elastic net. Fig. below shows coefficients paths using all 3 methods: ridge (Fig A, alpha=0), lasso (Fig B; alpha=1) and elastic net (Fig ...
123 views

### Techniques for analyzing ratios

I am looking for advice and comments that deal with the analysis of ratios and rates. In the field in which I work analysis of ratios in particular is widespread but I have read a few papers that ...
578 views

### Multiple imputation questions for multiple regression in SPSS

I am currently running a multiple regression model using imputed data and have a few questions. Background: Using SPSS 18. My data appears to be MAR. Listwise deletion of cases leaves me with only ...
231 views

### Modeling a spatial trend by regression with the $(x,y)$ coordinates as predictors

I plan to include coordinates as covariates in the regression equation in order to adjust for the spatial trend that exists in the data. After that, I want to test residuals on spatial autocorrelation ...
544 views

### Time series with multiple subjects and multiple variables

I am a web developer and novice statistician. My data looks something like this ...
94 views

### Bound for Arithmetic Harmonic mean inequality for matrices?

NOTE: This question has originally been posted in MSE, but it did not generate any interest. It was first posted there, because the question itself is a pure matrix-algebra question. Nevertheless, ...
511 views

### How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way ...
2k views

### How to choose between ANOVA and ANCOVA in a designed experiment?

I am conducting an experiment which has the following: DV: Slice consumption (continuous or could be categorical) IV: Healthy message, unhealthy message, no message (control) (3 groups in which ...
211 views

### Some of my predictors are on very different scales - do I need to transform them before fitting a linear regression model?

I would like to run linear regression over a multi-dimensional data set. There exist differences among different dimensions in terms of their magnitude of order. For instance, dimension 1 generally ...
9k views

### What is the correct way to test for significant differences between coefficients?

I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set ...
2k views

### Why are rlm() regression coefficient estimates different than lm() in R?

I am using rlm in the R MASS package to regress a multivariate linear model. It works well for a number of samples but I am getting quasi-null coefficients for a particular model: ...
2k views

### How to do a generalized linear model with multiple dependent variables in R?

I have six dependent variables (count data) and several independent variables, I see that in a MMR the script goes like this: ...
2k views

### Fitting multiple linear regression in R: autocorrelated residuals

I'm trying to estimate a multiple linear regression in R with an equation like this: regr <- lm(rate ~ constant + askings + questions + 0) askings and ...
179 views

### Linear relationship between explanatory variables in multiple regression

I was reading the multiple regression chapter of Data Analysis and Graphics Using R: An Example-Based Approach and was a bit confused to find out that it recommends checking for linear relationships ...
232 views

### What is an unbiased estimate of population R-square?

I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression. On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
1k views

### Multivariate linear regression vs neural network?

It seems that it is possible to get similar results to a neural network with a multivariate linear regression in some cases, and multivariate linear regression is super fast and easy. Under what ...
1k views

### Why use control variables in differences-in-differences?

I have a question on the differences-in-differences approach with the following standard equation: $$y= a + b_1\text{treat}+ b_2\text{post} + b_3\text{treat}\cdot\text{post} + u$$ where treat is a ...
260 views

### Help me fit this non-linear multiple regression that has defied all previous efforts

EDIT: Since making this post, I have followed up with an additional post here. Summary of the text below: I am working on a model and have tried linear regression, Box Cox transformations and GAM but ...
120 views

### Knapsack problem with uncertain profits

I am trying to find the optimal strategy for a game where the goal is to pick a number of items where the profit is uncertain but the weights are set. Each round of picking is a specific point in ...