# Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

223 questions
Filter by
Sorted by
Tagged with
107k views

### When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
5k views

### Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable. My questions are ...
173k views

### What is the difference between linear regression on y with x and x with y?

The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
47k views

### Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
21k views

### Why is polynomial regression considered a special case of multiple linear regression?

If polynomial regression models nonlinear relationships, how can it be considered a special case of multiple linear regression? Wikipedia notes that "Although polynomial regression fits a nonlinear ...
258k views

### What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
106k views

### PCA and proportion of variance explained

In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a ...
9k views

### Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
1k views

### Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
27k views

### Testing for linear dependence among the columns of a matrix

I have a correlation matrix of security returns whose determinant is zero. (This is a bit surprising since the sample correlation matrix and the corresponding covariance matrix should theoretically be ...
8k views

### Do we need gradient descent to find the coefficients of a linear regression model?

I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
10k views

### Fast linear regression robust to outliers

I am dealing with linear data with outliers, some of which are at more the 5 standard deviations away from the estimated regression line. I'm looking for a linear regression technique that reduces the ...
88k views

### What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
25k views

### Shape of confidence interval for predicted values in linear regression

I have noticed that the confidence interval for predicted values in an linear regression tends to be narrow around the mean of the predictor and fat around the minimum and maximum values of the ...
64k views

77k views

### Assumptions of linear models and what to do if the residuals are not normally distributed

I am a little bit confused on what the assumptions of linear regression are. So far I checked whether: all of the explanatory variables correlated linearly with the response variable. (This was the ...
2k views

### Where do the assumptions for linear regression come from? [duplicate]

I'v already known that there are several assumpations when using linear regression model. But I cannot understand why some of them exists. They are: independent errors normal distribution of errors ...
13k views

### Regression modelling with unequal variance

I would like to fit a linear model (lm) where the residuals variance is clearly dependent on the explanatory variable. The way I know to do this is by using glm with the Gamma family to model the ...
1k views

### Normality assumption in linear regression

As an assumption of linear regression, the normality of the distribution of the error is sometimes wrongly "extended" or interpreted as the need for normality of the y or x. Is it possible to ...
2k views

### Recovering raw coefficients and variances from orthogonal polynomial regression

It seems that if I have a regression model such as $y_i \sim \beta_0 + \beta_1 x_i+\beta_2 x_i^2 +\beta_3 x_i^3$ I can either fit a raw polynomial and get unreliable results or fit an orthogonal ...
29k views

### The main effect will be non-significant if the interaction is significant? [duplicate]

I am using linear mixed models to identify important factors, and it turns out that: A: significant B: not significant ...
946 views

### Identity in Simple Linear Model

I'm working on this identity $$\sum_{i-1}^n (y_i - \hat {\beta_0} - \hat {\beta_1}x_i)^2 = \sum_{i=1}^n y_i^2 - \hat {\beta_0}\sum_{i=1}^n y_i - \hat {\beta_1} \sum_{i=1}^n x_iy_i$$ I have these ...
16k views

### (Why) do overfitted models tend to have large coefficients?

I imagine that the larger a coefficient on a variable is, the more ability the model has to "swing" in that dimension, providing an increased opportunity to fit noise. Although I think I've got a ...
3k views

9k views

### For linear classifiers, do larger coefficients imply more important features?

I'm a software engineer working on machine learning. From my understanding, linear regression (such as OLS) and linear classification (such as logistic regression and SVM) make a prediction based on ...
44k views

### When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
8k views

### Understanding QR Decomposition

I've got a worked example (in R), that I'm trying to understand further. I'm using Limma to create a linear model and I'm trying to understand what's happening step by step in the fold change ...
7k views

### Perform linear regression, but force solution to go through some particular data points

I know how to perform a linear regression on a set of points. That is, I know how to fit a polynomial of my choice, to a given data set, (in the LSE sense). However, what I do not know, is how to ...
27k views

### VIF, condition Index and eigenvalues

I am currently assessing multicollinearity in my datasets. What threshold values of VIF and condition index below/above suggest a problem? VIF: I have heard that VIF $\geq 10$ is a problem. After ...
1k views

### Likelihood in Linear Regression

I am trying to understand how people derive the Likelihood for simple linear regression. Lets say that we just have one feature x and the outcome y. I do not doubt the expression with the normal ...
26k views

### General Linear Model vs. Generalized Linear Model (with an identity link function?)

This is my first post, so please take it easy on me if I am not following some standards! I did a search for my question and nothing came up. My question relates mostly around the practical ...
15k views

### Distinction between linear and nonlinear model

I have read some explanations about the properties of linear vs nonlinear models, but still I am sometimes not sure if a model on hand is a linear or a nonlinear one. For example, is the following ...
2k views

### Classic linear model - model selection

I have a classic linear model, with 5 possible regressors. They are uncorrelated with one another, and have quite low correlation with the response. I have arrived at a model where 3 of the regressors ...
5k views

### Should the difference between control and treatment be modelled explicitly or implicitly?

Given the following experimental setup: Multiple samples are taken from a subject and each sample is treated multiple ways (including a control treatment). What is mainly interesting is the ...
8k views

### Matrix notation for logistic regression

In linear regression (squared loss), using matrix we have a very concise notation for the objective $$\text{minimize}~~ \|Ax-b\|^2$$ Where $A$ is the data matrix, $x$ is the coefficients, and $b$ ...
994 views

### Linear vs. nonlinear regression

I have a set of values $x$ and $y$ which are theoretically related exponentially: $y = ax^b$ One way to obtain the coefficients is by applying natural logarithms in both sides and fitting a linear ...
1k views

### Is the linearity assumption in linear regression merely a definition of $\epsilon$?

I am revising linear regression. The textbook by Greene states: Now, of course there will be other assumptions on the linear regression model, such as $E(\epsilon|X)=0$. This assumption ...
7k views

### Interpret Regression Coefficients After various Differencing

There are few explanations I can find that describe how to interpret linear regression coefficients after differencing a time series (to eliminate a unit root). Is it just so simple that there is no ...
4k views

### How can I use the value of $R^2$ to test the linearity assumption in multiple regression analysis?

The below graphs are residual scatter plots of a regression test for which "normality", "homoscedasticity" and "independence" assumptions have already been met for sure! For testing the "linearity" ...
5k views

### What makes mean square error so good? [duplicate]

Our statistical inference course material states the following: The principle of mean square error can be derived from the principle of maximum likelihood (after we set a linear model where ...