Refers to any model where the a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

1
vote
1answer
23 views

Rearrange regression equation that includes a dummy variable

This is my regression equation: $10 = 5.44 + 0.26X_1 - 3.19X_2$ $X_2$ is a dummy predictor with two levels. Assume that the value of $X_2$ is 1 therefore regression equation is: $10 = 5.44 + ...
0
votes
0answers
34 views

Linear regression for classification

Suppose, I have a classification problem with 2 classes (0 and 1) and evaluation criteria is AUC. I used the following method: fit a linear regression and then pass its predictions through the ...
0
votes
1answer
12 views

normalizing predictor by another predictor

I'm fitting a linear model with outcome $Y$. I have measurements for variables $X_1$ and $X_2$. I hypothesize that $X_1$ and $Y$ are linearly related. I want to know the slope and significance of ...
0
votes
1answer
38 views

Matrix Inversion Error

I a Multiple linear regression model, from published literature, I am implementing a spreadsheet to generate new predictions based on the published model. the literature stated Coefficients and the ...
0
votes
0answers
35 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
0
votes
3answers
75 views

Econometrics - which independent variable has the greatest impact on dependent variable? [closed]

So I have a model predicting college GPA (dependent variable). I have around 10 independent variables, ranging from hours studied per week to alcoholic drinks consumed per week. That being said, how ...
1
vote
1answer
33 views

How to determine whether a dataset can be learned by Logistic regression?

As far as I know, Logistic Regression can deal with data in which positive and negative samples can be separated by a linear hyperplane. But if the data cannot be separated by a hyperplane, it cannot ...
2
votes
1answer
51 views

How does the distinction between association and causation affect the interpretation of linear models? [closed]

Lurking variables probably have something to do with this. I'm just trying to figure out how their difference can affect a linear model.
1
vote
0answers
19 views

nested multilevel model for differential expression analysis

I have read several other postings regarding nested models, but they did not seem to exactly capture my particular case, and I'm a bit unsure how to proceed with analysis of my model. Any help would ...
0
votes
0answers
11 views

The asymptotic slope of the BLP

So I am given that $X$ is a binary r.v. and the following assignments, $E[X] = A, E[Y|X=1]=B, E[Y|X=0]=C, E[Y^{2}|X=1] = D, E[Y^{2}|X=0]=E$. I must express my answers in terms of these expressions ...
0
votes
0answers
11 views

Use of Proc Mixed to find if product version has effect of its sales

I would like to confirm what I am doing is correct or not. I have the following data: Day Units sold (Var = Units) Number of stores in which the product was sold (var = stores) Version of the ...
0
votes
1answer
61 views

What are the benefits and disadvantages to Lasso, Ridge, Elastic Net, and Non Negative Garrotte Regularization techniques?

I am implementing these four regularization techniques for linear regression of stock data in MATLAB but i noticed elastic net is just the sum of Ridge and Lasso, and i dont full understand how ...
1
vote
1answer
23 views

Covariance of OLS estimator and residual = 0. Where is the mistake?

$Cov(b,e|X)$, where $b$ is the OLS estimator of the coefficients, $e$ is the residual vector, and $X$ is the regressor matrix. We know that $Cov(b,e|X)=E(be'|X)-E(b|X)E(e'|X)$ where ' $'$ ' is the ...
6
votes
2answers
98 views

Regression with inverse independent variable

Let's suppose I have a $N$-vector $Y$ of dependent variables, and an $N$-vector $X$ of independent variable. When $Y$ is plotted against $\frac{1}{X}$, I see that there is a linear relationship ...
4
votes
3answers
152 views

$R^2$ of linear regression with no variation in the response variable

Suppose I wish to fit $\hat{y} = \beta_0 + \beta_1x$ where the the data is as follows: x = 0.0, 0.1, 0.2, 0.3, 0.4 y = 0.0, 0.0, 0.0, 0.0, 0.0 Clearly, ...
0
votes
1answer
35 views

Are level 1 and level 2 residuals in a mixed effects model always normally distributed?

Take this mixed effects model: $y_{ij} = \beta_0 + \beta_1X_{ij} + \mu_{j} + \epsilon_{ij}$ The level 2 residuals are $\mu_{j}$ and the level 1 residuals are $\epsilon_{ij}$. As I understand the ...
1
vote
2answers
42 views

Influence of correlation in linear regression

I have an output $Y$ and some input values $X_1, \dots X_p$, where the number of variables are smaller than the number of observations ($p<<n$). I want to understand which of the variables have ...
2
votes
1answer
35 views

Residuals perfectly symmetric about zero against fitted values

Consider a modelling a response $Y$ against two categorical variables (which can take $4\times 2=8$ possible combinations). We have 16 values for the response, with two values for every combination of ...
1
vote
1answer
63 views

Linear model trace of the hat matrix in R

This question is about the difference between the sum of lm.influence(model)$hat and the trace of the Hat-Matrix $H := X (X' X)^{-1} X'$ calculated "by hand". ...
0
votes
1answer
33 views

likelihood in bayesian linear regression

I was going through the derivation for the likelihood of Bayesian linear regression http://en.wikipedia.org/wiki/Bayesian_linear_regression#Posterior_distribution I did not understand this step where ...
1
vote
1answer
8 views

Mixed effects model with level 2 explanatory variable

Take this linear mixed effects model, which is discussed on the CMM website: Centre for Multilevel Modelling $y_{ij} = \beta_0 + \beta_1X_{ij} + \beta_2\bar{X}_j + u_j + e_{ij}$ The variable $X$ is ...
0
votes
1answer
35 views

Which regression model to choose? [duplicate]

I have two models, one lm(y ~ x1 + x2 + 0) which gives me a close to 0.90 something $R^2$ and another model lm(y ~ x1 + x2) ...
3
votes
1answer
44 views

Post-hoc after GLM: What does it exactly say?

Background: I have been asked to model the change of weight of a few animals undergoing experimentation via a simple GLM (General Linear Model). The data looks something like this. Note that all data ...
0
votes
0answers
29 views

How can I create a linear regression model with some negative coefficients in R? [duplicate]

What I'm trying to do is to construct a linear model in a form like $$ Y = \beta_0X_0-\beta_1X_1+\beta_2X_2 + \beta_3 $$ where $\beta_0$, $\beta_1$ and $\beta_2$ are coefficient of predictors $X_0$, ...
1
vote
2answers
62 views

Negative fitted values in OLS regression

I am running a regression where my dependent variable is a cross-section of variances. Therefore, I require my predicted values (fitted values) to be positive. However, when running a simple OLS ...
0
votes
1answer
32 views

The effect of the order of observations on the distribution of $\hat{\beta}$ in Linear Regression

Consider linear regression. It is known that if $Y \sim N_n\left(X\beta, \sigma^2 I_n\right)$, where $X$ is $n \times p$ of rank $p$, then $$ \hat{\beta} \sim N_p\left(\beta, ...
0
votes
1answer
30 views

$\hat{\beta}^{(M)}_i\sim \hat{\beta}^{(N)}_i$ for linear regression?

Consider an i.i.d. sample $(X_1, Y_1), \dots, (X_N, Y_N)$, where each $X_i$ and $Y_i$ are $n$-dimensional column vectors, let $M \leq N$ and denote by $\hat{\beta}^{(M)}$ and $\hat{\beta}^{(N)}$ the ...
1
vote
1answer
27 views

Linear regression with redundant features (perfect multicolinearity)

Suppose $X \sim N(0,1)$, $Z=X$, and $Y=X$. An ordinary least squares regression problem is solved: $min_{(b1,b2)} \|Y-(b1*X+b_2*Z)\|_{2}^2$ This is a strictly convex function which must have a ...
2
votes
1answer
52 views

How to use a linear model with two factors and repeated measures?

Suppose I have a date set of the form: ...
9
votes
2answers
181 views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, ...
0
votes
0answers
22 views

regarding skip the intercept term once it is not statistically significant [duplicate]

After building the regression model, the intercept value is not statistically significant Is that reasonable to just skip it in the final regression model?
0
votes
0answers
21 views

Should I do this ARMA model?

These are the autocorrelations: As one can see, it is quite low around 0.02 for the first lag. But it is significantly nonzero, as the blue lines indicate. However, I dont think it makes sense to ...
0
votes
0answers
39 views

Methodological advice

I need some help validating the statistical methodology I'm trying to use to analyze some data. My data is from a repeated measures study where each participant did two activities. Half of the ...
2
votes
0answers
42 views

Regression with non-zero mean errors

I want to fit a linear regression model of the type $$y_j= x^{\top}_j\beta +\epsilon_j,\,\,\, j=1,\dots,n,$$ However, the distribution I am using for modelling $\epsilon_j$ does not have mean zero, ...
1
vote
1answer
35 views

Hypothesis test for the response variable in a least squares regression model

I have an equation where time it takes to get to work is based on time it takes to depart, number of red lights hit, and number of trains you encounter. The model is shown below: ...
1
vote
0answers
38 views

What is the best practices/way to do a linear regression on highly correlated variables

I wish to create a composite variable of a number of highly correlated variables. Each one of these variables contains different information and is useful in it's own right. But they are all highly ...
1
vote
0answers
67 views

testing interaction terms in regression model [duplicate]

Based on domain knowledge and preliminary variable selection, we have decided a set of 10 variables as predictor variables for building regression models. What are the general approaches to identify ...
4
votes
2answers
133 views

How to know if “best fit line” really represents known set of data?

I have a known set of data. I have created a "linear best fit line" for that set of data. Is there a way to determine how well my set of data fit that best fit line (some sort of score)? I'm very ...
2
votes
0answers
59 views

How to fit OLS with many categorical levels, on more than one category

This question is not meant to be a software question, but I will illustrate the issue using R a bit. My Understanding of the Simple Case If I have a simple linear model with a categorical variable ...
1
vote
0answers
16 views

Does additive Adjust-R square indicates variables in two models are independent?

If the adjust-R square for model Y ~ x1 + x2 +... xn is 0.11 and model Y ~ z1 + z2 +... zn is 0.07 and model Y ~ x1 + x2 +... xn + z1 + z2 +... zn is 0.18 Can I draw the conclusion that the object ...
1
vote
0answers
51 views

Combine several different sets of Linear Square Monte Carlo (LSMC) or Model Average

I am doing a project similar to LSMC (Linear Square Monte Carlo) for prediction. A Monte Carlo simulation engine is used to produce results, and a linear model is built on the same inputs and ...
3
votes
1answer
61 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
3
votes
1answer
77 views

Studentized residuals undefined

I am wondering if anyone could explain why there are some states where Studentized residuals are undefined. For example I got the following R code: ...
1
vote
0answers
38 views

R-squared adj. in multiple linear regression of 75% = high correlation?

I have a response column and a column of categorical predictors (around 25 categories) and I get with minitab linear regression analysis a R-sqr adjusted of 75%. ...
0
votes
0answers
17 views

Incorporating kernel into multiple regression

Let's say I have predictors $ \{x_1, x_2, ..., x_m, ... x_p\} $. I want to fit a multiple regression using $\{x_1,...,x_m\}$, but give more weight to points that are close to a particular $\vec{x}^*$ ...
1
vote
0answers
15 views

Probability denisty around a linear regression line

In a very basic problem or linear regression y= mx + q one can define an Confidence Interval around the line. This has a characteristic shape: narrow at the center getting bigger at the extremities. ...
0
votes
0answers
34 views

Regarding analysis of regression result and vif result

I am working on building a regression model. There are 51 points. The number of predictor variables is 37. The following is the result of running lm result. When trying to detecting the ...
2
votes
1answer
51 views

Why make the distribution of a variable more symmetric?

One of the goals of re-expressing data values is to "make the distribution of a variable (as seen its histogram, for example) more symmetric. My question is: why is more symmetric data better for ...
2
votes
0answers
13 views

What is the effect of simple transformations to predictor and/or response variables to the correlation constant (r)?

Context: A study of living conditions in 55 large U.S. cities found the mean January temperature (degrees Fahrenheit), altitude (feet above sea level), and latitude (degrees north of the equator). ...
1
vote
2answers
69 views

Is there a good recent Literature Review on Linear Regression models?

The literature review should include: Ordinary least squares (OLS) Generalized least squares (GLS) Least absolute deviation (LAD) Quantile regression Least-angle regression Ridge regression ...