Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
2
votes
0answers
8 views

Alternative to plug-in estimation for log-tranformed linear model

I want to estimate a relationship of the form: $$y=ax^b\times\epsilon$$ If I log this model i get: $$\log(y)=\log(a)+b\log(x)+ \log(\epsilon)$$ If I then proceed and estimate this model using a ...
0
votes
0answers
12 views

What properties does a set of data has to have in order to apply matrix multiplication method to find the optimal regression line using least square?

In the problem of linear regression, we are given $n$ observations $\{ (x_1, y_1),\dots,(x_n, y_n)\}$, where each input $x_i$ is a $d$-dimensional vector. Our goal is to estimate a linear predictor $f(...
3
votes
1answer
24 views

Difference between Multivariate Regression vs Iterative Regression on Residuals [duplicate]

Suppose one has an n × 2 matrix X (the independent variables) and a n × 1 vector y (the dependent variable). In a standard multiple linear regression setting, we solve for the 2 × 1 beta vector that ...
1
vote
0answers
31 views

Difference between predictions of the OLS model and a leave-one-out model

Consider OLS regression with the true model $y = {\theta^{*}}^{\text{T}} x + \varepsilon$, where $x$ denotes the (deterministic) independent variables, $y$ denotes the dependent (random) variable, and ...
0
votes
0answers
17 views

Distribution of a quadratic form [on hold]

Problem: Let $ x \sim N_{k}(\mu, \sigma^2 I)$. Show that $x^\prime x/\sigma ^2 \sim \chi^2(k, \mu ^\prime \mu/2\sigma ^2)$ I am trying to use the following theorem result but I am kinda confused how ...
3
votes
1answer
26 views

Why cannot the model $\frac {y_{i,j }} {N_{i,j } } = \beta_0 + \beta_1 X_i + e_{i,j }, \ y_{i,j}\sim B(N_{i,j},\pi_i)$ have constant variance?

The following example is taken from a book by Walter Stroup on Generalized linear mixed models, and are supposed to show some limitations on trying to write models in equation form. Let $y_{i,j } \...
2
votes
1answer
28 views

How to find the best fit line in this case?

Suppose that I have some data like this: There are $n$ data points $(x_i,y_i)$ and associated with each point are standard errors, $\sigma_{xi}$ and $\sigma_{yi}$ each with confidence level of $\sim ...
2
votes
2answers
26 views

Perfect multicollinearity with a cubic term in the model?

I'm trying to figure out why adding a cubic term in the model doesn't guarantee a perfect multicollinearity. If $X$ is known, then $X^3$ is known in both magnitude and sign and vice versa. It may not ...
0
votes
1answer
33 views

Value of $\sum_{j=1} (y_{j} - \bar{y})$ and proving properties of hat value

The i-th fitted value $\hat{Z}_i$ is written as a linear amalgam of response values $\hat{Z}_i=\sum_{j=1}h_{ij}Z_j$ where $h_{ij}=\frac{1}{n}+\frac{(y_i-\bar{y})(y_j-\bar{y})}{S_{yy}}$ and $S_{yy}=\...
0
votes
0answers
14 views

Interpretation of related lower-order interaction when higher-order interaction is significant

How do you investigate lower-order interactions which include one factor that is part of a significant higher-order interaction? Let's assume a research design with 4 factors (A, B, C, D), each with ...
0
votes
1answer
21 views

In a linear model, how is correlation among the independent variables related to uncertainty in the model coefficients? [closed]

Suppose I have a linear model Y=AX, and I tune A based on observed data. I know that correlation among my independent variables, X, will increase the uncertainty in my model coefficients, A. How do I ...
0
votes
0answers
11 views

Unbiasedness and Variance of Predictions

Here is the problem I'm working on: I'm not quite sure if I'm showing either unbiasedness property right, and am stuck on finding the expressions for the variances. Here's what I've done so far. (a) ...
0
votes
1answer
14 views

What is considered as parameters/coefficients/weights?

I am doing lesson 2 of the fast-ai course, and I find myself with a doubt about the course. Are the weights/parameters only the slope and the cut of axis y of a simple linear function $y = ax+b$? ...
1
vote
1answer
39 views

Linear regression for multi-class classification

Linear regression can be used for binary classification where it competes with logistic regression. While the fitted values from linear regression are not restricted to lie between 0 and 1, unlike ...
0
votes
0answers
5 views

Linear Dependency

In the context of matrices, how would one explain the notion of 'linear dependency between two or more variables' in an intuitive way? Would that imply near-perfect or perfect correlation amongst ...
2
votes
1answer
47 views

Linear regression formula interpretation in term of correlation

I have been taught that the optimal coefficients (in the MMSE sense) may be obtained by looking where the gradient of the associated loss is zero : With $ L(D,\beta) = ||X\beta-Y||^2 $ : $$ \frac{\...
1
vote
1answer
32 views

Does treating trial number as a continuous variable for linear models lose information?

If I create a linear model where Trial number is one of the predictors, am I losing any information by treating it as continuous (when in fact it is actually discrete + ordinal)? I believe the answer ...
0
votes
0answers
23 views

Question regarding linear regression weighting matrix

Consider the linear regression model $$b = Xy + e, \quad E[e] = 0, \quad E[ee'] = V$$ Assume that the matrix $X$ has linearly independent columns. It is well known that the minimum variance affine ...
0
votes
0answers
4 views

Is it okay to control for the initial level of a variable when predicting a delta score in a regression model?

I have a repeated measures design where over the course of weeks participant's are repeatedly engaging in an exercise that consists of reporting an emotion score, following some instructions intended ...
0
votes
1answer
36 views

Linear Model Assumptions

Hi, above is the plot of residuals against the fitted value of a linear model. I am asked to determine if the assumptions for the linear model hold in this case. I don't think the constant variance ...
0
votes
1answer
25 views

Nested leave-one-out for parameter selection

Using the following linear regression model workflow, I was able to generate a model that was robust to LOOCV. Because of posts such as this, I know that feature selection should be done inside the ...
0
votes
0answers
18 views

Proof of distribution of $Y^{T}AY$

I am reading a proof that given $Y \sim N(\mu, \Sigma)$, where $\Sigma$ is positive definite, $Y^{T}AY \sim \chi^{2}_{p}(\mu^{T}A\mu)$ iff $A \Sigma A =A$ and $A$ has rank $p$. One of the steps in ...
0
votes
1answer
36 views

Estimation of prediction confidence interval

I have a linear model $Y=\hat{\alpha}X+\hat{\beta}$ fit on a bunch of samples $(X_i,Y_i)$. How can I compute the prediction confidence interval for a given $X'$ ? Assuming than $X'$ does not belong to ...
3
votes
1answer
101 views

How do I define an “interaction” contrast with single explantory variable?

Suppose I have an experiment with four groups defined by two binary variables. ...
1
vote
0answers
17 views

Linear model theory online resources

I was wondering if you could tell me some online videos or other resources to learn theory of linear models. I am a visual learner. I learn more from video lectures. My focus is to learn: Matrices, ...
2
votes
0answers
29 views

How to predict the standard deviation in linear regression?

When linear regression is formulated probabilistically using MLE, turns out that what we used to get as output, is actually the mean of $P(y|x)$, the latter is normal distributed around the correct ...
0
votes
0answers
27 views

When averaging models, which models need to meet assumptions?

I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1
vote
0answers
41 views

Is my residual scatter-plot normal? can I still run a linear regression?

Am I still able to run a regression if my residuals look like these? This has happened to all my variables: it's equally distributed but very systematic. Is there a way i can fix this? My p-p plots ...
-1
votes
0answers
29 views

Need a help on interpreting Linear regression outputs of two models - with and without interaction

How to interpret the output following 2 regression models? Why AHI.x and BMI is not significant in the ...
1
vote
1answer
24 views

What is the process of obtaining Var(βhat) in simple linear regression?

I have just started statistics and we have used the estimation strategy OLS to obtain the parameter estimate of the independent variable for a simple linear regression model. As I understand it, my ...
0
votes
1answer
45 views

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
0
votes
1answer
14 views

How to draw LDA decision boundary given a fixed covariance matrix

I have been searching all over the internet and have not found an answer that specifically answers my question, so I apologize in advance if this has already been answered somewhere else and I did not ...
0
votes
1answer
22 views

Data level aggregation for price elasticity

I have just started working on price elasticity and I have some fundamental question. Question 1: I have daily price and sales data for a product, now if I want to calculate the price elasticity ...
0
votes
0answers
7 views

Difference of 'Adjusted eGFR change' between each category and the reference category

Dear StackExchange users, I am struggling to understand "the Difference in eGFR change" in the following table and why the value does not simply equal or close to the difference of 'Adjusted eGFR ...
0
votes
0answers
15 views

Linear regression on a non-negative independent variable

I'm trying to model the independent variable "Age" based on some features. If I construct my model simply as; Age ~ intercept + feat1 + feat2 Then Age can be ...
1
vote
0answers
28 views

Why do subset selection linear models have higher variance than the full model?

I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods. By retaining a subset of ...
0
votes
0answers
31 views

confidence contours for linear model with multiple dependent variables

A book I have on regression analysis describes a technique for determining confidence contours of the parameters of a linear model $$ Y^{\textrm{model}} = f(\boldsymbol{x}, \boldsymbol{\theta}) = \...
1
vote
1answer
53 views

Could data be described by a straight line when Pearson Correlation Coefficient has the highest absolute values?

Suppose that there is a dataset of 2D points $(x_i, y_i)$, consider the following statement: "When the Pearson Correlation Coefficient(PCC) between $x$ and $y$ is equal to -1 or 1 (highest absolute ...
2
votes
2answers
65 views

Why does the linear regression algorithm assume the input residuals (errors) to be normal distributed? [duplicate]

I am trying to know the assumptions of linear regression (LR). I understand linear regression needs the relationship between the independent and dependent variables to be linear, but LR also assumes ...
0
votes
0answers
13 views

Using a subset of parameters in joint confidence region of a linear model

For a standard linear model of the form $y = X\beta + \epsilon$, where $\beta$ is a vector of parameters. we can calculate an individual confidence interval for each parameter (of 1-$\alpha$ quartile)....
0
votes
0answers
18 views

Two different ways to write variance of OLS beta

I know that for OLS, we can write $var(\hat{\beta}) = \sigma^2 (X^{T}X)^{-1}$. Then for a the last variable $p$, we have $var(\hat{\beta}_p) = \frac{\sigma^2}{\langle x_p, x_p \rangle}$. However, we ...
1
vote
0answers
26 views

What is the variance of the estimator in ordinary least squares with correlated residuals

If we assumed that $y \sim N(X\beta,S)$ where S= $\sigma^2\begin{bmatrix} 1 & \rho & \rho &...\\ \rho & 1 & \rho &...\\ \rho & \rho & 1 &...\\ \rho & \rho &...
4
votes
1answer
32 views

Decreasing trend in residual vs fitted value plot

Here is my residual vs fitted value plot. It shows a decreasing trend. Can someone explain to me what could cause this to happen and how do I correct my model to produce a better fit? I am fitting ...
0
votes
1answer
32 views

Averaging individual predictions in a group

I created linear model to give prediction for a team member (individual). Can I use this model to give average (individual) prediction in a team by providing average values of features among team ...
0
votes
0answers
6 views

Predict with average values of dependent features

I created linear regression model to predict story points by individual team member (in sprint). Since story points are relative by sprint team, I trained my model after scaling story points to 0 - 1 ...
2
votes
0answers
45 views

Problem with calculating a confidence interval

I'm working on Exercise 3.2 from Elements of Statistical Learning. It asks to find a $95\%$ confidence interval for a linear regression prediction (ordinary least squares are used) using two different ...
1
vote
1answer
51 views

Screening candidate models before AIC comparison?

I am interested in identifying the best of 3 physiologically reasonable models that fits my continuous data. Data is some measure derived from neurons recorded from 3 adjacent regions of brain tissue (...
2
votes
0answers
30 views

Linear regression: an input variable as a multiplication/addition of other input variables

We have ways to identify collinearity in a multiple regression (using input variables' correlation matrix) and remove collinearity by dropping some of the collinear input variables. But what can be ...
1
vote
0answers
13 views

Using likelihood ratio test to compare 2 nested simple linear models instead of anova

Is it valid to use a likelihood ratio test to compare 2 nested linear models instead of anova? I'm trying to assess whether the quadratic model is a better fit to the data. I know anova seems to be ...
0
votes
0answers
12 views

Price elasticity for n data points

So let's say I have 100 data points which contains the price and sales of a product. Just to start I assume that the relationship is of the form Q = a - b*P where P ...