Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

1,584 questions
Filter by
Sorted by
Tagged with
4k views

How can I use the value of $R^2$ to test the linearity assumption in multiple regression analysis?

The below graphs are residual scatter plots of a regression test for which "normality", "homoscedasticity" and "independence" assumptions have already been met for sure! For testing the "linearity" ...
5k views

Some of my predictors are on very different scales - do I need to transform them before fitting a linear regression model?

I would like to run linear regression over a multi-dimensional data set. There exist differences among different dimensions in terms of their magnitude of order. For instance, dimension 1 generally ...
2k views

Linear model overfitting due to too many covariates

My study design involves a control and 2 test groups plus some covariates. Each group consists of around 20 observations. In total I look at around 1,000 variables. I created a linear model using the ...
2k views

Recovering raw coefficients and variances from orthogonal polynomial regression

It seems that if I have a regression model such as $y_i \sim \beta_0 + \beta_1 x_i+\beta_2 x_i^2 +\beta_3 x_i^3$ I can either fit a raw polynomial and get unreliable results or fit an orthogonal ...
1k views

How do you know which predictor value to use?

I'm trying to automate linear regression with R, although I don't really have a concrete background in statistics. I was wondering: Are there numerical techniques in determining whether the predictor ...
231 views

Under what conditions can a PLS regression model be expressed by single linear equation?

I am confused by two, yet inconsistent for me, facts: Since the PLS regression is expressed by matrices of scores and loadings as $$X=TP^T+E\\Y=UQ^T+F$$ how it can be translated into linear equation ...
1k views

How to establish relationships using box plots?

I am examining the relationship between a categorical variable and a quantitative one. For the test of linearity, I was told that I must use scatter plots. However, this is no longer applicable if one ...
5k views

Help with the Ljung-Box test for time independence of residuals

I fit a simple linear model $y = bX$ to a data set today, and that produced 24 residuals (I have 24 data points, one for each year from 1984-2007). I would like to test the time-independence of the ...
161 views

47k views

Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
282 views

Autocorrelated predictors in linear models

I need to predict the outcomes of a time-series variable $Y$ based on two time-series predictors $X1$ and $X2$. For simplicity I will only illustrate $X1$ in the rest of this question. The ...
9k views

Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
7k views

How to apply coefficient term for factors and interactive terms in a linear equation?

Using R, I have fitted a linear model for a single response variable from a mix of continuous and discrete predictors. This is uber-basic, but I'm having trouble grasping how a coefficient for a ...
712 views

Broken Tobit regressions

I have a dataset with 43,422 observations and a left-censored (at 0) dependent variable. Of the $n$ observations, 42,536 are left-censored and 886 are not. I plan on analyzing this with a Tobit ...
24k views

Linear model Heteroscedasticity

I have the following linear model: To address the residuals heteroscedasticity I have tried to apply a log transformation on the dependent variable as $\log(Y + 1)$ but I still see the same fan out ...
1k views

Are there problems with inference using linear regression on observational data with highly skewed distributions of predictor values?

I am using a linear regression model to perform inference on some observational data. The samples are from an observational study and highly skewed along some of the dummy variables in the regression. ...
349 views

Simple linear regression

Consider the simple regression model $y=\beta_0+\beta_1x+u$, where $\text{corr}(x,u)=1$ and all random variables have normal distributions. Is it possible to provide asymptotically consistent ...
174k views

What is the difference between linear regression on y with x and x with y?

The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
8k views

Can the interaction term of two insignificant coefficients be significant?

Lets say I have a linear regression with two numeric explanatory variables: A and B. Consider the following scenarios: A and B are both insignificant A is significant, B is insignificant; or the ...
107k views

PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
834 views

In simple linear regression, how do I show that the squared test statistic for the null hypothesis has an F-distribution?

In simple linear regression, $t = \frac{\hat\beta_1 - \beta_1}{\hat\sigma \sqrt{S_{xx}}}$ is the test statistic for the null hypothesis $H_0 : \beta_1 = 0$. How can I express $t^2$ as an F-...
1k views

Linear regression optimization

I have the following linear model with 2 predictors $x_1, x_2$ and a cubic transform on $x_1$: $$\hat{Y} = c +\beta_1x_1 + \beta_2x_1^2 + \beta_3x_1^3 + \beta_4x_2$$ Where $x_1$ can assume ...
324 views

Difference between linear regression prediction intervals and logistic regression targets

I am trying to understand the difference between logistic regression probabilities and linear regression prediction intervals. For example, let's say we have a database of student test scores in the ...
147 views

Accounting for the presence of a variable in regression?

So, as an example, let's say that I have data on a small auction that I conducted. Let's say I am selling a car and there are 5 bidders, and I have data on each of their bids, who won, who ...
3k views

Difference between ANCOVA and Hierarchical Regression

Is there a difference between ANCOVA (as performed under the 'General Linear Model (GLM)') and Hierarchical Regression (as performed under 'Regression') in SPSS? I am testing the main effects and ...
5k views

GLM on unbalanced design

I have a dataset that comprises 200 males and 250 females and I am testing their responses on the relationship between X and Y. X and Y are continuous and X1 (gender) is categorical. I am using ...
280 views

How to design a contrast matrix with combined levels for a categorical variable?

I wish to make a contrast matrix in the case of a linear model. I have one factor with three levels: T, N and ...
749 views

How to test a reduced linear model passing through the origin?

For simple ordinary linear regression $y=a+bx$, if I want to use the reduced model $y=bx$, i.e. passing through the origin, how to do statistical tests on the validity of such a linear model?
8k views

If I repeat every sample observation in a linear regression model and rerun the regression how would the result be affected?

Say I have N observations, possibly multiple factors and I repeat each observation twice (or M times) how would a regression on this new set of size NM compare to a regression on just the original ...
246 views

What does “arrived at linear multiple regression models” mean?

I'm reading a paper on a study where a number of respondents received questions which they graded using a scale of 1-5. In the conclusion, the authors wrote that they "arrived at linear multiple ...
2k views

Possible extensions to the default diagnostic plots for lm (in R and in general)?

I started digging a bit into the plot.lm function, this function gives six plots for lm, they are: a plot of residuals against fitted values a Scale-Location plot of sqrt(| residuals |) against ...
764 views

Notation for nested factors that are random variables in R

I need to compute the estimate the variance component for my data. I have the following model, y = gene, cell line, gene*cell line, DNA extract[cell line] How do I write this for the lme function? ...
211 views

Similar sum of squares in GLM

I have run the General Linear Model in SPSS to analyse the effect of several demographic variables (e.g. gender, age) on the relationship between X and Y. So essentially, this is an analysis to see ...
11k views

Significant interaction between covariate and factor in SPSS GLM

In testing gender difference on the relationship between variable A and B, A is the covariate (or independent variable) B is the dependent variable Gender is the factor As I understand it, if there ...
6k views

General linear model with interaction term in SPSS

Following my question here, I will be most grateful for further assistance to clear my confusion. The confusion is because I have read in many places that when you use ANCOVA in GLM (in SPSS), the ...
7k views

Explanation of “covariate” in general linear model in SPSS

I am completing a project for a client using general linear model (GLM command) in SPSS/PASW (Ver. 17) Basically, the project is designed to find out if factors such as gender and age affect the ...
2k views

Using Beta to interpret interaction in general linear model

Following the question here, someone suggested that I could just look at the B column in SPSS "Parameter Estimates" table to interpret the interaction. For instance, the B column for one of my tests ...
220 views

Significant interaction in linear model with Pearson r as possible explanation

I have tested gender difference on the relationship between Variable A and B using linear model (GLM command in SPSS). I have found that the interaction of A (covariate/independent variable) with ...
25k views

How to calculate the difference of two slopes?

Is there a method to understand if two lines are (more or less) parallel? I have two lines generated from linear regressions and I would like to understand if they are parallel. In other words, I ...