Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
1
vote
1answer
22 views

Interaction term is significant WITHOUT main affects… AND main effects are significant WITHOUT interaction term?

I am trying to determine the effect of a person's weight and the incline that they are running over on their running speed. I'm just using a simple linear model in R, but I get a weird situation where ...
0
votes
1answer
27 views

Doesn't adding a quadratic term in Linear Reg Model violate independence of predictors (Multicollinearity)?

I was going through an example of Polynomial regression and could not understand why adding quadratic term doesn't violate Linear Model assumption of multi-collinearity as we are just squaring the ...
0
votes
0answers
21 views

Relationship between two linear regression models

This is a homework question from my statistics course about regression, dummy variables. It provides the information below, Firstly, a dummy variable is defined ...
0
votes
2answers
90 views

Meaning of $\hat{\beta}$ of the linear regression model [closed]

In the simple linear regression model, $\hat{\beta}$ is the sum of independent normally distributed random variables. Is it false because in linear regression there is $\beta$ and not $\hat{\beta}$?
0
votes
4answers
77 views

Can least-squares linear regression ever produce no solution at all?

Is it ever possible for least-squares linear regression (linear in both features and weights) NOT to produce a solution? That is, after we set each partial derivative to zero, can the resulting system ...
3
votes
2answers
130 views

Is the assumption of normality of the error term needed to use p-value?

I have been thinking lately about the following: 1. Is the normality assumption of the error term really needed in order to make use of p-values for linear regression models? A previous CV post (...
1
vote
2answers
20 views

Resources suggestion about linear model

I was wondering if you could tell me about some self-learning resources for linear model theories. My professor has been using "A First Course in Linear Model Theory, Ravishanker and Dey. Publisher: ...
2
votes
0answers
9 views

Alternative to plug-in estimation for log-tranformed linear model

I want to estimate a relationship of the form: $$y=ax^b\times\epsilon$$ If I log this model i get: $$\log(y)=\log(a)+b\log(x)+ \log(\epsilon)$$ If I then proceed and estimate this model using a ...
0
votes
0answers
13 views

What properties does a set of data has to have in order to apply matrix multiplication method to find the optimal regression line using least square?

In the problem of linear regression, we are given $n$ observations $\{ (x_1, y_1),\dots,(x_n, y_n)\}$, where each input $x_i$ is a $d$-dimensional vector. Our goal is to estimate a linear predictor $f(...
3
votes
1answer
33 views

Difference between Multivariate Regression vs Iterative Regression on Residuals [duplicate]

Suppose one has an n × 2 matrix X (the independent variables) and a n × 1 vector y (the dependent variable). In a standard multiple linear regression setting, we solve for the 2 × 1 beta vector that ...
1
vote
0answers
35 views

Difference between predictions of the OLS model and a leave-one-out model

Consider OLS regression with the true model $y = {\theta^{*}}^{\text{T}} x + \varepsilon$, where $x$ denotes the (deterministic) independent variables, $y$ denotes the dependent (random) variable, and ...
4
votes
1answer
39 views

Why cannot the model $\frac {y_{i,j }} {N_{i,j } } = \beta_0 + \beta_1 X_i + e_{i,j }, \ y_{i,j}\sim B(N_{i,j},\pi_i)$ have constant variance?

The following example is taken from a book by Walter Stroup on Generalized linear mixed models, and are supposed to show some limitations on trying to write models in equation form. Let $y_{i,j } \...
2
votes
1answer
31 views

How to find the best fit line in this case?

Suppose that I have some data like this: There are $n$ data points $(x_i,y_i)$ and associated with each point are standard errors, $\sigma_{xi}$ and $\sigma_{yi}$ each with confidence level of $\sim ...
2
votes
2answers
33 views

Perfect multicollinearity with a cubic term in the model?

I'm trying to figure out why adding a cubic term in the model doesn't guarantee a perfect multicollinearity. If $X$ is known, then $X^3$ is known in both magnitude and sign and vice versa. It may not ...
0
votes
1answer
38 views

Value of $\sum_{j=1} (y_{j} - \bar{y})$ and proving properties of hat value

The i-th fitted value $\hat{Z}_i$ is written as a linear amalgam of response values $\hat{Z}_i=\sum_{j=1}h_{ij}Z_j$ where $h_{ij}=\frac{1}{n}+\frac{(y_i-\bar{y})(y_j-\bar{y})}{S_{yy}}$ and $S_{yy}=\...
0
votes
0answers
17 views

Interpretation of related lower-order interaction when higher-order interaction is significant

How do you investigate lower-order interactions which include one factor that is part of a significant higher-order interaction? Let's assume a research design with 4 factors (A, B, C, D), each with ...
0
votes
1answer
21 views

In a linear model, how is correlation among the independent variables related to uncertainty in the model coefficients? [closed]

Suppose I have a linear model Y=AX, and I tune A based on observed data. I know that correlation among my independent variables, X, will increase the uncertainty in my model coefficients, A. How do I ...
0
votes
0answers
13 views

Unbiasedness and Variance of Predictions

Here is the problem I'm working on: I'm not quite sure if I'm showing either unbiasedness property right, and am stuck on finding the expressions for the variances. Here's what I've done so far. (a) ...
0
votes
1answer
14 views

What is considered as parameters/coefficients/weights?

I am doing lesson 2 of the fast-ai course, and I find myself with a doubt about the course. Are the weights/parameters only the slope and the cut of axis y of a simple linear function $y = ax+b$? ...
1
vote
1answer
67 views

Linear regression for multi-class classification

Linear regression can be used for binary classification where it competes with logistic regression. While the fitted values from linear regression are not restricted to lie between 0 and 1, unlike ...
0
votes
0answers
5 views

Linear Dependency

In the context of matrices, how would one explain the notion of 'linear dependency between two or more variables' in an intuitive way? Would that imply near-perfect or perfect correlation amongst ...
2
votes
1answer
49 views

Linear regression formula interpretation in term of correlation

I have been taught that the optimal coefficients (in the MMSE sense) may be obtained by looking where the gradient of the associated loss is zero : With $ L(D,\beta) = ||X\beta-Y||^2 $ : $$ \frac{\...
1
vote
1answer
35 views

Does treating trial number as a continuous variable for linear models lose information?

If I create a linear model where Trial number is one of the predictors, am I losing any information by treating it as continuous (when in fact it is actually discrete + ordinal)? I believe the answer ...
0
votes
0answers
30 views

Linear regression: equivalence of forms of the minimum variance, affine unbiased estimator

Background Consider the linear regression model: $$y = X\beta + e\\E[e] = 0 \quad E[ee^T] = V$$ It is well known that the minimum variance affine unbiased estimator (MVAUE) of $\beta$ exists if and ...
0
votes
0answers
4 views

Is it okay to control for the initial level of a variable when predicting a delta score in a regression model?

I have a repeated measures design where over the course of weeks participant's are repeatedly engaging in an exercise that consists of reporting an emotion score, following some instructions intended ...
0
votes
1answer
36 views

Linear Model Assumptions

Hi, above is the plot of residuals against the fitted value of a linear model. I am asked to determine if the assumptions for the linear model hold in this case. I don't think the constant variance ...
0
votes
1answer
29 views

Nested leave-one-out for parameter selection

Using the following linear regression model workflow, I was able to generate a model that was robust to LOOCV. Because of posts such as this, I know that feature selection should be done inside the ...
0
votes
0answers
20 views

Proof of distribution of $Y^{T}AY$

I am reading a proof that given $Y \sim N(\mu, \Sigma)$, where $\Sigma$ is positive definite, $Y^{T}AY \sim \chi^{2}_{p}(\mu^{T}A\mu)$ iff $A \Sigma A =A$ and $A$ has rank $p$. One of the steps in ...
0
votes
1answer
49 views

Estimation of prediction confidence interval

I have a linear model $Y=\hat{\alpha}X+\hat{\beta}$ fit on a bunch of samples $(X_i,Y_i)$. How can I compute the prediction confidence interval for a given $X'$ ? Assuming than $X'$ does not belong to ...
3
votes
1answer
109 views

How do I define an “interaction” contrast with single explantory variable?

Suppose I have an experiment with four groups defined by two binary variables. ...
1
vote
0answers
19 views

Linear model theory online resources

I was wondering if you could tell me some online videos or other resources to learn theory of linear models. I am a visual learner. I learn more from video lectures. My focus is to learn: Matrices, ...
2
votes
0answers
32 views

How to predict the standard deviation in linear regression?

When linear regression is formulated probabilistically using MLE, turns out that what we used to get as output, is actually the mean of $P(y|x)$, the latter is normal distributed around the correct ...
0
votes
0answers
28 views

When averaging models, which models need to meet assumptions?

I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1
vote
0answers
43 views

Is my residual scatter-plot normal? can I still run a linear regression?

Am I still able to run a regression if my residuals look like these? This has happened to all my variables: it's equally distributed but very systematic. Is there a way i can fix this? My p-p plots ...
1
vote
1answer
31 views

What is the process of obtaining Var(βhat) in simple linear regression?

I have just started statistics and we have used the estimation strategy OLS to obtain the parameter estimate of the independent variable for a simple linear regression model. As I understand it, my ...
1
vote
1answer
48 views

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
0
votes
1answer
20 views

How to draw LDA decision boundary given a fixed covariance matrix

I have been searching all over the internet and have not found an answer that specifically answers my question, so I apologize in advance if this has already been answered somewhere else and I did not ...
0
votes
1answer
26 views

Data level aggregation for price elasticity

I have just started working on price elasticity and I have some fundamental question. Question 1: I have daily price and sales data for a product, now if I want to calculate the price elasticity ...
0
votes
0answers
7 views

Difference of 'Adjusted eGFR change' between each category and the reference category

Dear StackExchange users, I am struggling to understand "the Difference in eGFR change" in the following table and why the value does not simply equal or close to the difference of 'Adjusted eGFR ...
0
votes
0answers
23 views

Linear regression on a non-negative independent variable

I'm trying to model the independent variable "Age" based on some features. If I construct my model simply as; Age ~ intercept + feat1 + feat2 Then Age can be ...
1
vote
0answers
37 views

Why do subset selection linear models have higher variance than the full model?

I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods. By retaining a subset of ...
0
votes
0answers
34 views

confidence contours for linear model with multiple dependent variables

A book I have on regression analysis describes a technique for determining confidence contours of the parameters of a linear model $$ Y^{\textrm{model}} = f(\boldsymbol{x}, \boldsymbol{\theta}) = \...
1
vote
1answer
55 views

Could data be described by a straight line when Pearson Correlation Coefficient has the highest absolute values?

Suppose that there is a dataset of 2D points $(x_i, y_i)$, consider the following statement: "When the Pearson Correlation Coefficient(PCC) between $x$ and $y$ is equal to -1 or 1 (highest absolute ...
2
votes
2answers
96 views

Why does the linear regression algorithm assume the input residuals (errors) to be normal distributed? [duplicate]

I am trying to know the assumptions of linear regression (LR). I understand linear regression needs the relationship between the independent and dependent variables to be linear, but LR also assumes ...
0
votes
0answers
16 views

Using a subset of parameters in joint confidence region of a linear model

For a standard linear model of the form $y = X\beta + \epsilon$, where $\beta$ is a vector of parameters. we can calculate an individual confidence interval for each parameter (of 1-$\alpha$ quartile)....
0
votes
0answers
18 views

Two different ways to write variance of OLS beta

I know that for OLS, we can write $var(\hat{\beta}) = \sigma^2 (X^{T}X)^{-1}$. Then for a the last variable $p$, we have $var(\hat{\beta}_p) = \frac{\sigma^2}{\langle x_p, x_p \rangle}$. However, we ...
2
votes
1answer
41 views

What is the variance of the estimator in ordinary least squares with correlated residuals

If we assumed that $y \sim N(X\beta,S)$ where S= $\sigma^2\begin{bmatrix} 1 & \rho & \rho &...\\ \rho & 1 & \rho &...\\ \rho & \rho & 1 &...\\ \rho & \rho &...
4
votes
1answer
57 views

Decreasing trend in residual vs fitted value plot

Here is my residual vs fitted value plot. It shows a decreasing trend. Can someone explain to me what could cause this to happen and how do I correct my model to produce a better fit? I am fitting ...
0
votes
1answer
34 views

Averaging individual predictions in a group

I created linear model to give prediction for a team member (individual). Can I use this model to give average (individual) prediction in a team by providing average values of features among team ...
0
votes
0answers
8 views

Predict with average values of dependent features

I created linear regression model to predict story points by individual team member (in sprint). Since story points are relative by sprint team, I trained my model after scaling story points to 0 - 1 ...