Refers to any model where the a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

1
vote
1answer
57 views

Linear regression model mistakenly gives $R^2$ equal to 1

I'm using R to create a linear regression model from survey data about public sentiment for a new technology. I am encountering a problem where the addition of a new explanatory variable raises the ...
1
vote
1answer
21 views

Significance codes in linear model with factors

I am setting up a linear model in R and need help understanding the significance codes when one of my independent variables is a factor - i.e., dummy variable for each possible value For a scalar ...
3
votes
0answers
26 views

Prior for the coefficients of a linear regression model

I have a linear regression model $\bf Y=\bf{X}\bf{\beta}+\epsilon$. I want to assign a prior on $\bf\beta$ in order to derive the posterior predictive model $p(y_{predictive}|\bf{y},\bf{X},\beta)$. ...
1
vote
0answers
33 views

Is a model including a square root of a variable linear in the parameters? [duplicate]

Is the model $$ y = \gamma_0 + \gamma_1 + \sqrt x + \varepsilon $$ linear in parameters? ( $\varepsilon$ is the error term.)
0
votes
0answers
14 views

Linear regression with faster decrease in coefficient error/variance?

Suppose we have set of variables $Y$ and $X$, which know are related by a linear relation $y_i=\alpha x_i +\beta$, and important for us is to find $\alpha$ and $\beta$ and the error in estimating ...
3
votes
2answers
67 views

Is it an assumption of the normal linear model that explanatory variables are uncorrelated with the errors?

Some books seem to include an assumption for the normal linear model which I have never seen before. They say that there must be no correlation between between the explanatory variables and the ...
1
vote
0answers
14 views

How to do contrasts with weighted observations in R's linear model function lm()

As part of a simple simulation study, I have the following lines of code in R: ...
0
votes
0answers
4 views

Adjacent-period regression and time dummy

I have a sample of products' prices and their attributes across three brands to estimate price change. I have used linear adjacent-period regression with time dummies to capture average price change ...
2
votes
1answer
49 views

Endogeneity test instrumental variables

I'm reading a paper in which is used the following endogeneity test: First of all, we have the initial linear model: $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + e$$ $x_3$ is the ...
3
votes
1answer
35 views

Handling outliers in Bayesian linear regression

I am reading this post which talks about Robust Linear regression in a Bayesian setting. The particular blog post can be found here: http://twiecki.github.io/blog/2013/08/27/bayesian-glms-2/ There ...
0
votes
1answer
26 views

Why does adding more terms into a linear model always increase the r-squared value?

Many statistics textbooks state that adding more terms into a linear model always reduces the sum of squares and in turn increases the r-squared value. This has led to the use of the adjusted ...
0
votes
0answers
25 views

Interaction between continuous predictor and repeated-measures factor in R

How can I test for the presence of an interaction between a categorical repeated-measures factor and a continuous predictor? I'm using R. My data looks like this: ...
1
vote
2answers
56 views

How do I read this linear model output from R? [duplicate]

I normally use SPSS for my statistics, however after having some issues with violations I've had to try and run a linear model in R as apparently its more robust. Someone sent me the code that I ...
2
votes
0answers
23 views

Two univariate vs. multivariate analysis

Suppose I have 2 response variables $Y_1,Y_2$ and some predictor variables $x_j$. Is it the same if I use 2 separate linear models, one for each of the response variables, vs. using a multivariate ...
1
vote
1answer
43 views

Statistically significant difference in linear regression model predictions of the mean values

In my academic report I have a task to check whether or not mean values (for given two predictor values) predicted by the simple linear regression model are "statictically significantly different". I ...
4
votes
3answers
140 views

Transforming a variable when original variable does not have explantory power

Sometimes in multivariate linear regression, there will be one explanatory variable that does not contribute much in way of explanatory power. Then, we will perform a tranform on that variable, i.e ...
2
votes
1answer
58 views

Why take transpose of regressor variable in linear regression

I am stuck trying to understand the basic calculation of ordinary least squares. From wikipedia $$y = \beta X^T + \varepsilon$$ where $X$ is the independent variable, $Y$ is the dependent variable ...
2
votes
1answer
45 views

Sum of Square decomposition

Question about the Total, Explained, and Residual Sum of Squares. I am in the simple linear regression model. Could you help me clarify why the residual sum of squares (SSE where E stands for errors) ...
3
votes
1answer
52 views

Is it possible to seed RANSAC with a given line?

I am analyzing a stream of data and I want to seed every new instance with the best guess output (line) of the previous, so as to eventually converge. Given that Scikit Learn - RANSAC is an iterative ...
1
vote
0answers
36 views

Principal component regression on polynomial terms

One of the data sets I am working upon had 3 variables which were having almost 100% correlation among themselves. Since I am learning regression modelling I thought I'll do principal component ...
0
votes
0answers
25 views

Robust Standard Errors(SE) estimators vs SE estimators assuming Conditionally Homoskedasticity [duplicate]

If both the asymptotic Variance-Covariance matrix estimators (robust and non-robust) are consistent to the same matrix, i.e., both will have the same efficiency (True?), then what is the advantage of ...
5
votes
1answer
105 views

How to prove that Asymptotic Variance-Covariance matrix of OLS estimator is Positive Definite?

I'm trying to understand why the asymptotic Variance-Covariance($\text{Avar}(b)$) matrix of OLS estimator is Positive Definite(PD), like it's stated in Hayashi's book on page 113. We know that ...
2
votes
1answer
43 views

Can you add up effect sizes of two related variables (e.g. age + age$^2$) from a single regression model?

I am interested in the effect of age on outcome Y. I have two nested linear regression models to test linear and quadratic effects of age: Y= $\beta_0$ + $\beta_1$ some_covariate + $\beta_2$ Age + ...
1
vote
1answer
34 views

Confidence interval for a multiple of regression coefficient

I am trying to model relationship between length of stay of patients in hospital(Y) vs Age in years(X). The data set I've got doesn't specify the unit of length of stay. So now estimated value of my ...
0
votes
1answer
18 views

MSEP and R2pred for Linear Model

I have two set of data 1-Training (Calibrating) 2-Test. With these datasets, I Fit the model using first dataset. predict using the second dataset x-variables I have to test the closeness of the ...
1
vote
1answer
28 views

Rearrange regression equation that includes a dummy variable

This is my regression equation: $10 = 5.44 + 0.26X_1 - 3.19X_2$ $X_2$ is a dummy predictor with two levels. Assume that the value of $X_2$ is 1 therefore regression equation is: $10 = 5.44 + ...
0
votes
0answers
38 views

Linear regression for classification

Suppose, I have a classification problem with 2 classes (0 and 1) and evaluation criteria is AUC. I used the following method: fit a linear regression and then pass its predictions through the ...
0
votes
1answer
13 views

normalizing predictor by another predictor

I'm fitting a linear model with outcome $Y$. I have measurements for variables $X_1$ and $X_2$. I hypothesize that $X_1$ and $Y$ are linearly related. I want to know the slope and significance of ...
0
votes
1answer
42 views

Matrix Inversion Error

I a Multiple linear regression model, from published literature, I am implementing a spreadsheet to generate new predictions based on the published model. the literature stated Coefficients and the ...
0
votes
0answers
47 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
1
vote
1answer
40 views

How to determine whether a dataset can be learned by Logistic regression?

As far as I know, Logistic Regression can deal with data in which positive and negative samples can be separated by a linear hyperplane. But if the data cannot be separated by a hyperplane, it cannot ...
2
votes
1answer
53 views

How does the distinction between association and causation affect the interpretation of linear models? [closed]

Lurking variables probably have something to do with this. I'm just trying to figure out how their difference can affect a linear model.
1
vote
0answers
27 views

nested multilevel model for differential expression analysis

I have read several other postings regarding nested models, but they did not seem to exactly capture my particular case, and I'm a bit unsure how to proceed with analysis of my model. Any help would ...
0
votes
0answers
11 views

The asymptotic slope of the BLP

So I am given that $X$ is a binary r.v. and the following assignments, $E[X] = A, E[Y|X=1]=B, E[Y|X=0]=C, E[Y^{2}|X=1] = D, E[Y^{2}|X=0]=E$. I must express my answers in terms of these expressions ...
0
votes
0answers
12 views

Use of Proc Mixed to find if product version has effect of its sales

I would like to confirm what I am doing is correct or not. I have the following data: Day Units sold (Var = Units) Number of stores in which the product was sold (var = stores) Version of the ...
0
votes
1answer
67 views

What are the benefits and disadvantages to Lasso, Ridge, Elastic Net, and Non Negative Garrotte Regularization techniques?

I am implementing these four regularization techniques for linear regression of stock data in MATLAB but i noticed elastic net is just the sum of Ridge and Lasso, and i dont full understand how ...
1
vote
1answer
23 views

Covariance of OLS estimator and residual = 0. Where is the mistake?

$Cov(b,e|X)$, where $b$ is the OLS estimator of the coefficients, $e$ is the residual vector, and $X$ is the regressor matrix. We know that $Cov(b,e|X)=E(be'|X)-E(b|X)E(e'|X)$ where ' $'$ ' is the ...
6
votes
2answers
103 views

Regression with inverse independent variable

Let's suppose I have a $N$-vector $Y$ of dependent variables, and an $N$-vector $X$ of independent variable. When $Y$ is plotted against $\frac{1}{X}$, I see that there is a linear relationship ...
4
votes
3answers
184 views

$R^2$ of linear regression with no variation in the response variable

Suppose I wish to fit $\hat{y} = \beta_0 + \beta_1x$ where the the data is as follows: x = 0.0, 0.1, 0.2, 0.3, 0.4 y = 0.0, 0.0, 0.0, 0.0, 0.0 Clearly, ...
0
votes
1answer
40 views

Are level 1 and level 2 residuals in a mixed effects model always normally distributed?

Take this mixed effects model: $y_{ij} = \beta_0 + \beta_1X_{ij} + \mu_{j} + \epsilon_{ij}$ The level 2 residuals are $\mu_{j}$ and the level 1 residuals are $\epsilon_{ij}$. As I understand the ...
1
vote
2answers
45 views

Influence of correlation in linear regression

I have an output $Y$ and some input values $X_1, \dots X_p$, where the number of variables are smaller than the number of observations ($p<<n$). I want to understand which of the variables have ...
2
votes
1answer
39 views

Residuals perfectly symmetric about zero against fitted values

Consider a modelling a response $Y$ against two categorical variables (which can take $4\times 2=8$ possible combinations). We have 16 values for the response, with two values for every combination of ...
2
votes
1answer
94 views

Linear model trace of the hat matrix in R

This question is about the difference between the sum of lm.influence(model)$hat and the trace of the Hat-Matrix $H := X (X' X)^{-1} X'$ calculated "by hand". ...
0
votes
1answer
34 views

likelihood in bayesian linear regression

I was going through the derivation for the likelihood of Bayesian linear regression http://en.wikipedia.org/wiki/Bayesian_linear_regression#Posterior_distribution I did not understand this step where ...
1
vote
1answer
13 views

Mixed effects model with level 2 explanatory variable

Take this linear mixed effects model, which is discussed on the CMM website: Centre for Multilevel Modelling $y_{ij} = \beta_0 + \beta_1X_{ij} + \beta_2\bar{X}_j + u_j + e_{ij}$ The variable $X$ is ...
0
votes
1answer
40 views

Which regression model to choose? [duplicate]

I have two models, one lm(y ~ x1 + x2 + 0) which gives me a close to 0.90 something $R^2$ and another model lm(y ~ x1 + x2) ...
3
votes
1answer
50 views

Post-hoc after GLM: What does it exactly say?

Background: I have been asked to model the change of weight of a few animals undergoing experimentation via a simple GLM (General Linear Model). The data looks something like this. Note that all data ...
0
votes
0answers
29 views

How can I create a linear regression model with some negative coefficients in R? [duplicate]

What I'm trying to do is to construct a linear model in a form like $$ Y = \beta_0X_0-\beta_1X_1+\beta_2X_2 + \beta_3 $$ where $\beta_0$, $\beta_1$ and $\beta_2$ are coefficient of predictors $X_0$, ...
1
vote
2answers
68 views

Negative fitted values in OLS regression

I am running a regression where my dependent variable is a cross-section of variances. Therefore, I require my predicted values (fitted values) to be positive. However, when running a simple OLS ...
0
votes
1answer
32 views

The effect of the order of observations on the distribution of $\hat{\beta}$ in Linear Regression

Consider linear regression. It is known that if $Y \sim N_n\left(X\beta, \sigma^2 I_n\right)$, where $X$ is $n \times p$ of rank $p$, then $$ \hat{\beta} \sim N_p\left(\beta, ...