Tagged Questions

Refers to any model where the a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

0
votes
1answer
19 views

The effect of the order of observations on the distribution of $\hat{\beta}$ in Linear Regression

Consider linear regression. It is known that if $Y \sim N_n\left(X\beta, \sigma^2 I_n\right)$, where $X$ is $n \times p$ of rank $p$, then $$ \hat{\beta} \sim N_p\left(\beta, ...
0
votes
1answer
18 views

$\hat{\beta}^{(M)}_i\sim \hat{\beta}^{(N)}_i$ for linear regression?

Consider an i.i.d. sample $(X_1, Y_1), \dots, (X_N, Y_N)$, where each $X_i$ and $Y_i$ are $n$-dimensional column vectors, let $M \leq N$ and denote by $\hat{\beta}^{(M)}$ and $\hat{\beta}^{(N)}$ the ...
0
votes
0answers
8 views

Linear regression with redundant features (perfect multicolinearity)

Suppose $X \sim N(0,1)$, $Z=X$, and $Y=X$. An ordinary least squares regression problem is solved: $min_{(b1,b2)} \|Y-(b1*X+b_2*Z)\|_{2}^2$ This is a strictly convex function which must have a ...
1
vote
1answer
24 views

How to use a linear model with two factors and repeated measures?

Suppose I have a date set of the form: ...
9
votes
2answers
122 views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, ...
0
votes
0answers
22 views

regarding skip the intercept term once it is not statistically significant [duplicate]

After building the regression model, the intercept value is not statistically significant Is that reasonable to just skip it in the final regression model?
0
votes
0answers
20 views

Should I do this ARMA model?

These are the autocorrelations: As one can see, it is quite low around 0.02 for the first lag. But it is significantly nonzero, as the blue lines indicate. However, I dont think it makes sense to ...
0
votes
0answers
38 views

Methodological advice

I need some help validating the statistical methodology I'm trying to use to analyze some data. My data is from a repeated measures study where each participant did two activities. Half of the ...
2
votes
0answers
40 views

Regression with non-zero mean errors

I want to fit a linear regression model of the type $$y_j= x^{\top}_j\beta +\epsilon_j,\,\,\, j=1,\dots,n,$$ However, the distribution I am using for modelling $\epsilon_j$ does not have mean zero, ...
1
vote
1answer
26 views

Hypothesis test for the response variable in a least squares regression model

I have an equation where time it takes to get to work is based on time it takes to depart, number of red lights hit, and number of trains you encounter. The model is shown below: ...
0
votes
0answers
20 views

the reason that adding one more predictor variables will cause lm model to be all NA

I have a data set with multiple predictor variable candidates, and try to experiment with different combinations. During the experiment process, I tried using the first 9 variables, ...
1
vote
0answers
31 views

What is the best practices/way to do a linear regression on highly correlated variables

I wish to create a composite variable of a number of highly correlated variables. Each one of these variables contains different information and is useful in it's own right. But they are all highly ...
1
vote
0answers
55 views

testing interaction terms in regression model [duplicate]

Based on domain knowledge and preliminary variable selection, we have decided a set of 10 variables as predictor variables for building regression models. What are the general approaches to identify ...
4
votes
2answers
118 views

How to know if “best fit line” really represents known set of data?

I have a known set of data. I have created a "linear best fit line" for that set of data. Is there a way to determine how well my set of data fit that best fit line (some sort of score)? I'm very ...
2
votes
0answers
52 views

How to fit OLS with many categorical levels, on more than one category

This question is not meant to be a software question, but I will illustrate the issue using R a bit. My Understanding of the Simple Case If I have a simple linear model with a categorical variable ...
1
vote
0answers
15 views

Does additive Adjust-R square indicates variables in two models are independent?

If the adjust-R square for model Y ~ x1 + x2 +... xn is 0.11 and model Y ~ z1 + z2 +... zn is 0.07 and model Y ~ x1 + x2 +... xn + z1 + z2 +... zn is 0.18 Can I draw the conclusion that the object ...
1
vote
0answers
21 views

Combine several different sets of Linear Square Monte Carlo (LSMC) or Model Average

I am doing a project similar to LSMC (Linear Square Monte Carlo) for prediction. A Monte Carlo simulation engine is used to produce results, and a linear model is built on the same inputs and ...
2
votes
0answers
30 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
3
votes
1answer
70 views

Studentized residuals undefined

I am wondering if anyone could explain why there are some states where Studentized residuals are undefined. For example I got the following R code: ...
1
vote
0answers
29 views

R-squared adj. in multiple linear regression of 75% = high correlation?

I have a response column and a column of categorical predictors (around 25 categories) and I get with minitab linear regression analysis a R-sqr adjusted of 75%. ...
0
votes
0answers
16 views

Incorporating kernel into multiple regression

Let's say I have predictors $ \{x_1, x_2, ..., x_m, ... x_p\} $. I want to fit a multiple regression using $\{x_1,...,x_m\}$, but give more weight to points that are close to a particular $\vec{x}^*$ ...
1
vote
0answers
15 views

Probability denisty around a linear regression line

In a very basic problem or linear regression y= mx + q one can define an Confidence Interval around the line. This has a characteristic shape: narrow at the center getting bigger at the extremities. ...
0
votes
0answers
24 views

Regarding analysis of regression result and vif result

I am working on building a regression model. There are 51 points. The number of predictor variables is 37. The following is the result of running lm result. When trying to detecting the ...
2
votes
1answer
40 views

Why make the distribution of a variable more symmetric?

One of the goals of re-expressing data values is to "make the distribution of a variable (as seen its histogram, for example) more symmetric. My question is: why is more symmetric data better for ...
2
votes
0answers
10 views

What is the effect of simple transformations to predictor and/or response variables to the correlation constant (r)?

Context: A study of living conditions in 55 large U.S. cities found the mean January temperature (degrees Fahrenheit), altitude (feet above sea level), and latitude (degrees north of the equator). ...
1
vote
2answers
63 views

Is there a good recent Literature Review on Linear Regression models?

The literature review should include: Ordinary least squares (OLS) Generalized least squares (GLS) Least absolute deviation (LAD) Quantile regression Least-angle regression Ridge regression ...
1
vote
2answers
106 views

Linear regression including categorical variables with hundreds of levels

I am trying to teach myself data science by solving some of the problems available on the internet. Currently I am trying to predict a fraud event with the aid of 4 categorical variables. Each of the ...
2
votes
0answers
55 views

Is regularization of a linear model really needed?

I'm going to do linear regression on a data set with 60k observations, each with 120 features. The way I see it, there is no why in the world that with more then 50 samples per dimension, a linear low ...
2
votes
1answer
166 views

Why not using cross validation for estimating the error of a linear model?

Cross validation (CV) seems to be a simple and useful tool for estimating the out-of-sample error of a linear regression model, even though it is rarely used for this purpose. Why that? Is there a ...
1
vote
0answers
27 views

$R^2$ (coefficient of determination) and linearity in multiple linear regression

For simple linear regression (SLR), in order for $R^2$ (the coefficient of determination) to be a meaningful measure, it must be true that $X$ and $Y$ are linearly correlated. Specifically, $R^2=r^2$, ...
6
votes
2answers
305 views

Why is GLM different than an LM with transformed variable

As explained in this course handout (page 1), a linear model can be written in the form: $$ y = \beta_1 x_{1} + \cdots + \beta_p x_{2} + \varepsilon_i$$ , where $y$ is the response variable and ...
0
votes
0answers
11 views

selecting genes specific/agnostic to condition

I have a microarray transcriptomics experiment. The design of which is something like this : ...
0
votes
0answers
29 views

Given a covariance matrix from a Linear regression, how do I calculate the standard error of the coefficients?

I have an OLS with autocorrelation in the residuals. I'm using python statsmodels, and found that there is the sandwich_covariance matrix, which can cal Reference to Newey-West covariance matrix: ...
0
votes
1answer
51 views

Non-linear Model vs Linear Model for a dataset

I have a time series dataset for a city. The dataset contains rainfall amount and the number of repairman requests to a company. The company has 20 shops in different blocks of city and the rainfall ...
0
votes
0answers
23 views

Troubles reporting transformed variables for log and sqrt into a general equation

Good morning everybody, I see CrossValidated has really high level of questions and answer; I am just a student so I hope this question is not too basic... Suggestion of further readings available ...
1
vote
0answers
43 views

Linear model / models analysis

Above are three plots of the Linear model I am trying to analyze. The first one is a basic plot of the linear data: ...
1
vote
0answers
28 views

What is Linear Projection [closed]

Can anyone help explain the linear projection and its application? In Wooldgridge's textbook of Econometrics, he introduced the basics of linear project, which I understand what it is but I still ...
0
votes
1answer
29 views

Modelling interaction

How does adding interaction term in the model adjust for it or why do we need to add interaction? I am working on logistic regression model with treatment and race as predictors. I have added ...
12
votes
3answers
476 views

How to discuss a scatterplot with multiple emerging lines?

We have measured two variables, and the scatterplot seems to suggest multiple "linear" models. Is there a way to try to distill those models? Identifying other independent variables has turned out to ...
0
votes
0answers
25 views

How to improve linear model generalization when autocorrelation is present?

I have features $X_t$ and response $Y_t$ (all continuous variables) and my objective is to find the best estimate of $f(X_t)=Y_t$ where $f$ is linear, and 'best' is defined as lowest generalisation ...
1
vote
1answer
37 views

LSmeans - Unbalanced data with interactions

I wish to analyze an unbalanced data set with 3 variables Tleaf, Tair, and orientation (factor with two levels). Considering the effect of the factor "orientation", I wish to determine if "Tair" has a ...
1
vote
1answer
42 views

What is the benefit of knowing the F statistic in multiple linear regression?

One of the basic figures you get when running multiple linear regression using almost any off-the-shelf software is the F statistics. However, I cannot recall one situation, where the F value was low ...
2
votes
1answer
153 views

Is a linear model OK?

I carried out a linguistics experiment where I gave a text for people to comment on. I recorded and transcribed their comments. I would like use the number of utterances per sentence in the text they ...
2
votes
0answers
57 views

Show alias coefficient of Plackett Burman design equals $r_{ij}=\frac{c_{1i}'c_{2j}}{N}$

Consider a Plackett Burman design with N rows and let $\beta_{1}$ be the of the regression coefficients corresponding to the main effects and let $\beta_{2}$ be the vector of the regression coefficients ...
0
votes
0answers
27 views

Drop1() and Summary() on lm object

I need to analyse unbalanced data through linear regression: modJuin=lm(TleafMax~TairMax*orientation, na.action="na.exclude", data=aJuin) "TairMax" is a ...
0
votes
0answers
29 views

How to compute/run LDA with 3 classes

I couldn't find one example on how to compute LDA with 3 classes (nor what is the algorithm). for example i have the following observations and classes: (each observation in one-dimensional) $ ...
1
vote
1answer
37 views

Identifiability in linear regression and time series

The multivariate linear regression model is given by $\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$, where $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0, ...
3
votes
1answer
106 views

How can I optimize coefficients of an arbitrary model?

This might be terribly easy but I'm probably lacking the keywords to search for. Assume the following (dummy) data: ...
2
votes
1answer
50 views

Multiple Linear Regression: Obtaining a Stable Model

I am working with a data set of ~1200 rows and 60 variables, and I'm trying to build a multiple linear regression model. I do this by separating 10% of the dataset to be used for validation and I use ...
0
votes
0answers
30 views

Fuzzy regression (using linear programming)

I want to replicate a fuzzy regression using a linear programming problem approach. I have the following information: " A fuzzy regression analysis with only one independent variable X results in ...