Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
13
votes
2answers
4k views

How can I use the value of $R^2$ to test the linearity assumption in multiple regression analysis?

The below graphs are residual scatter plots of a regression test for which "normality", "homoscedasticity" and "independence" assumptions have already been met for sure! For testing the "linearity" ...
9
votes
2answers
5k views

Some of my predictors are on very different scales - do I need to transform them before fitting a linear regression model?

I would like to run linear regression over a multi-dimensional data set. There exist differences among different dimensions in terms of their magnitude of order. For instance, dimension 1 generally ...
3
votes
1answer
2k views

Linear model overfitting due to too many covariates

My study design involves a control and 2 test groups plus some covariates. Each group consists of around 20 observations. In total I look at around 1,000 variables. I created a linear model using the ...
14
votes
1answer
2k views

Recovering raw coefficients and variances from orthogonal polynomial regression

It seems that if I have a regression model such as $y_i \sim \beta_0 + \beta_1 x_i+\beta_2 x_i^2 +\beta_3 x_i^3$ I can either fit a raw polynomial and get unreliable results or fit an orthogonal ...
-1
votes
3answers
1k views

How do you know which predictor value to use?

I'm trying to automate linear regression with R, although I don't really have a concrete background in statistics. I was wondering: Are there numerical techniques in determining whether the predictor ...
4
votes
1answer
231 views

Under what conditions can a PLS regression model be expressed by single linear equation?

I am confused by two, yet inconsistent for me, facts: Since the PLS regression is expressed by matrices of scores and loadings as $$X=TP^T+E\\Y=UQ^T+F$$ how it can be translated into linear equation ...
1
vote
0answers
1k views

How to establish relationships using box plots?

I am examining the relationship between a categorical variable and a quantitative one. For the test of linearity, I was told that I must use scatter plots. However, this is no longer applicable if one ...
4
votes
2answers
5k views

Help with the Ljung-Box test for time independence of residuals

I fit a simple linear model $y = bX$ to a data set today, and that produced 24 residuals (I have 24 data points, one for each year from 1984-2007). I would like to test the time-independence of the ...
2
votes
0answers
161 views

What is the distribution of the 'achieved' $R^2$?

I am interested in the distribution of/performing inference on the 'achieved' $R^2$ coefficient in multiple linear regression. Suppose that $y\sim x\beta + \epsilon$ with $\epsilon \sim \mathcal{N}\...
122
votes
3answers
258k views

What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
2
votes
2answers
4k views

How to compute significant interaction estimates when main effect is not significant?

I have a linear model of a dependent variable, $y$, with two predictor variables, year and site, and their interaction, with year being numeric and site categorical. The main effect of year is not ...
4
votes
3answers
266 views

Applying an interaction term to all the IVs

I have a linear model with 6 IVs and would like to analyze the effect of an interaction term applied to all the IVs. To illustrate, let's say we're predicting the Win/Loose ratio of NBA basketball ...
1
vote
2answers
339 views

Testing for collinearity and parallelism with the axes

Cross-posted from math; let me know if you need me to wait there for a bit before posting here I have a set of datapoints in $\mathbb{R}^3$ (integer or real values of $X$, $Y$ and $Z$, for, say. 30 ...
2
votes
1answer
6k views

Categorical fixed effect w/ 3 levels in LMER

I have a categorical fixed effect with 3 levels that I'm trying to enter into an LME. ...
3
votes
0answers
182 views

Asymptotics of 0-1 classification loss

I am interested in training a simple binary linear classifier. That is, I will find a vector of weights $\bf w$ such that I can predict the class of new example by the sign of $f(x) = w^T x$. I ...
0
votes
0answers
243 views

What does the residuals effectively mean in this model?

I'm doing sme test using R and linear models. This is the code I'm using: ...
0
votes
1answer
87 views

Speed of linear modeling

Asked already on stackexchange, this place is probably a better fit, if somebody could delete the old post I'd appreciate it. I have a linear regression equation from school , which gives a value ...
4
votes
1answer
3k views

Linear separability for a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ $...
101
votes
2answers
47k views

Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
4
votes
0answers
282 views

Autocorrelated predictors in linear models

I need to predict the outcomes of a time-series variable $Y$ based on two time-series predictors $X1$ and $X2$. For simplicity I will only illustrate $X1$ in the rest of this question. The ...
15
votes
5answers
9k views

Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
9
votes
3answers
7k views

How to apply coefficient term for factors and interactive terms in a linear equation?

Using R, I have fitted a linear model for a single response variable from a mix of continuous and discrete predictors. This is uber-basic, but I'm having trouble grasping how a coefficient for a ...
3
votes
0answers
712 views

Broken Tobit regressions

I have a dataset with 43,422 observations and a left-censored (at 0) dependent variable. Of the $n$ observations, 42,536 are left-censored and 886 are not. I plan on analyzing this with a Tobit ...
10
votes
3answers
24k views

Linear model Heteroscedasticity

I have the following linear model: To address the residuals heteroscedasticity I have tried to apply a log transformation on the dependent variable as $\log(Y + 1)$ but I still see the same fan out ...
4
votes
1answer
1k views

Are there problems with inference using linear regression on observational data with highly skewed distributions of predictor values?

I am using a linear regression model to perform inference on some observational data. The samples are from an observational study and highly skewed along some of the dummy variables in the regression. ...
2
votes
2answers
349 views

Simple linear regression

Consider the simple regression model $y=\beta_0+\beta_1x+u$, where $\text{corr}(x,u)=1$ and all random variables have normal distributions. Is it possible to provide asymptotically consistent ...
97
votes
9answers
174k views

What is the difference between linear regression on y with x and x with y?

The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
7
votes
4answers
8k views

Can the interaction term of two insignificant coefficients be significant?

Lets say I have a linear regression with two numeric explanatory variables: A and B. Consider the following scenarios: A and B are both insignificant A is significant, B is insignificant; or the ...
90
votes
4answers
107k views

PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
1
vote
0answers
834 views

In simple linear regression, how do I show that the squared test statistic for the null hypothesis has an F-distribution?

In simple linear regression, $t = \frac{\hat\beta_1 - \beta_1}{\hat\sigma \sqrt{S_{xx}}} $ is the test statistic for the null hypothesis $H_0 : \beta_1 = 0$. How can I express $t^2$ as an F-...
4
votes
1answer
1k views

Linear regression optimization

I have the following linear model with 2 predictors $x_1, x_2$ and a cubic transform on $x_1$: $$ \hat{Y} = c +\beta_1x_1 + \beta_2x_1^2 + \beta_3x_1^3 + \beta_4x_2 $$ Where $x_1$ can assume ...
3
votes
1answer
324 views

Difference between linear regression prediction intervals and logistic regression targets

I am trying to understand the difference between logistic regression probabilities and linear regression prediction intervals. For example, let's say we have a database of student test scores in the ...
1
vote
2answers
147 views

Accounting for the presence of a variable in regression?

So, as an example, let's say that I have data on a small auction that I conducted. Let's say I am selling a car and there are 5 bidders, and I have data on each of their bids, who won, who ...
4
votes
1answer
3k views

Difference between ANCOVA and Hierarchical Regression

Is there a difference between ANCOVA (as performed under the 'General Linear Model (GLM)') and Hierarchical Regression (as performed under 'Regression') in SPSS? I am testing the main effects and ...
7
votes
2answers
5k views

GLM on unbalanced design

I have a dataset that comprises 200 males and 250 females and I am testing their responses on the relationship between X and Y. X and Y are continuous and X1 (gender) is categorical. I am using ...
4
votes
1answer
280 views

How to design a contrast matrix with combined levels for a categorical variable?

I wish to make a contrast matrix in the case of a linear model. I have one factor with three levels: T, N and ...
3
votes
1answer
749 views

How to test a reduced linear model passing through the origin?

For simple ordinary linear regression $y=a+bx$, if I want to use the reduced model $y=bx$, i.e. passing through the origin, how to do statistical tests on the validity of such a linear model?
14
votes
2answers
8k views

If I repeat every sample observation in a linear regression model and rerun the regression how would the result be affected?

Say I have N observations, possibly multiple factors and I repeat each observation twice (or M times) how would a regression on this new set of size NM compare to a regression on just the original ...
2
votes
2answers
246 views

What does “arrived at linear multiple regression models” mean?

I'm reading a paper on a study where a number of respondents received questions which they graded using a scale of 1-5. In the conclusion, the authors wrote that they "arrived at linear multiple ...
11
votes
2answers
2k views

Possible extensions to the default diagnostic plots for lm (in R and in general)?

I started digging a bit into the plot.lm function, this function gives six plots for lm, they are: a plot of residuals against fitted values a Scale-Location plot of sqrt(| residuals |) against ...
2
votes
2answers
764 views

Notation for nested factors that are random variables in R

I need to compute the estimate the variance component for my data. I have the following model, y = gene, cell line, gene*cell line, DNA extract[cell line] How do I write this for the lme function? ...
-1
votes
2answers
211 views

Similar sum of squares in GLM

I have run the General Linear Model in SPSS to analyse the effect of several demographic variables (e.g. gender, age) on the relationship between X and Y. So essentially, this is an analysis to see ...
2
votes
2answers
11k views

Significant interaction between covariate and factor in SPSS GLM

In testing gender difference on the relationship between variable A and B, A is the covariate (or independent variable) B is the dependent variable Gender is the factor As I understand it, if there ...
0
votes
2answers
6k views

General linear model with interaction term in SPSS

Following my question here, I will be most grateful for further assistance to clear my confusion. The confusion is because I have read in many places that when you use ANCOVA in GLM (in SPSS), the ...
2
votes
2answers
7k views

Explanation of “covariate” in general linear model in SPSS

I am completing a project for a client using general linear model (GLM command) in SPSS/PASW (Ver. 17) Basically, the project is designed to find out if factors such as gender and age affect the ...
-1
votes
1answer
2k views

Using Beta to interpret interaction in general linear model

Following the question here, someone suggested that I could just look at the B column in SPSS "Parameter Estimates" table to interpret the interaction. For instance, the B column for one of my tests ...
1
vote
1answer
220 views

Significant interaction in linear model with Pearson r as possible explanation

I have tested gender difference on the relationship between Variable A and B using linear model (GLM command in SPSS). I have found that the interaction of A (covariate/independent variable) with ...
9
votes
3answers
25k views

How to calculate the difference of two slopes?

Is there a method to understand if two lines are (more or less) parallel? I have two lines generated from linear regressions and I would like to understand if they are parallel. In other words, I ...
7
votes
3answers
12k views

How to store the standard errors with the lm() function in R? [closed]

All is in the title... I know how to store the estimates but I don't know how to store their standard errors... Thanks ...
1
vote
1answer
767 views

NAs in summary of linear model generated from significant genes for continuous response variable identified using SAM

I have a gene expression data-set with log2-transformed expression values (no NAs) for 495 genes for 59 samples for which values of a continuous response variable (r) are also known (no NAs). I want ...