Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

Filter by
Sorted by
Tagged with
0
votes
1answer
10 views

Testing reliability of regression coefficients

If I run a logit/linear regression for the purpose of measuring marginal effects and estimating the causal impact of a specific independent variable on the dependent variable, is there a reliable way ...
0
votes
0answers
12 views

Formulating the null hypothesis when the depended variable is a ratio involving the independent variable

I face the following simple regression model: $$ \widehat{rdexp}=\hat{\beta}_{0}+\hat{\beta}_{1}log(sales) $$ where $$ \widehat{rdexp}=\frac{RD}{sales} $$ and $RD$ is some positive number. I am having ...
0
votes
0answers
16 views

Demand Forecasting with Price Promotions

I'm trying to use Excel Solver and Linear Regression to forecast demand of a product with variables like different types of promotions and baseline level. I am looking at pooled regression but I don't ...
1
vote
0answers
29 views

Effect of L2 regularization on Linear Regression

I am starting with L2 regularization on linear regression. Case without regularization: objective function $(Xw-y)^T(Xw-y)$ parameters vector $w= (X^TX)^{-1}X^Ty.$ Case with regularization: ...
0
votes
1answer
26 views

Main effect vs simple effect

In the following linear model: $y = \beta_0 +\beta_1 x_1 +\beta_2 x_2 + \beta_3x_1x_2 + \epsilon$ what are called $x_i$? The term $x_1x_2$ is the interaction. In ANOVA, $x_i$ are named "main ...
0
votes
0answers
8 views

Interpreting categorical dummy variable esimtates in linear regression

I have a linear regresssion equation Y = b0X1 + b1X2 + b2 Both my X1 and X2 variables are categorical variables with multiple levels, and my regression fit gives ...
0
votes
1answer
68 views

Upper bound for variance of $\hat{\beta}$ in multiple linear regression

The variance of the beta estimator in an ordinary-least-squares multiple linear regression to express $Y$ as a (linear) function of $X$, $\hat{\beta}$, can be expressed as (knowing $X$ and $\sigma^2$ ...
-1
votes
0answers
54 views

Distribution of estimated variance of residuals in simple regression model [duplicate]

We have our simple regression estimated model: $$ Yi=b_{0}+b_{1}Xi $$ We know that the estimator for $\sigma^2$ is: $$ \hat\sigma^2=\frac{1}{n-2}\sum_{i=1}^{n}\varepsilon_i^2 $$ But how can we prove ...
0
votes
0answers
17 views

comment for the given data (linear regression)

An actuary is asked to check a linear regression calculation performed by a trainee. The trainee reports a least-squares slope parameter estimate of b ! = 13.7 and a sample correlation coefficient r = ...
1
vote
1answer
105 views

Overview of the linear model $Y = X \cdot \beta + \varepsilon$

My question regards the linear model $Y = X \cdot \beta + \varepsilon$. I currently attend a lecture on linear models and I realize this issue is very basic. In our script there are plenty of ...
0
votes
0answers
13 views

What are we plotting the residuals of a simple linear regression against if we want a normal probability plot? [duplicate]

I understand the interpretation of what occurs when we plot residuals for a normal probability plot, but I am trying to recollect/understand what we are plotting those residuals against? It would ...
0
votes
0answers
15 views

What would be the minimum data points between to known variables with fairly significant variations

There are two variables, daily steam and production of MWhs. The variation is significant day to day, month to month. What would be the minimum number or data points for linear regression to obtain ...
0
votes
0answers
6 views

combing a classification model with a smoothing model and a linear regression model

I would like to use a 1) classification model to predict the class probability. 2) I would like to 'smooth' the probabilities and use the smoothed probabilities as a predictor feature in a 3) linear ...
0
votes
0answers
21 views

How do I fit a multiple linear regression where I know the relationship between some of the coefficients

I want to fit a linear regression with multiple independent variables. Say that I know that the coefficient of one variable is twice as large as the coefficient of another variable, or that one ...
1
vote
0answers
37 views

Basis Functions with Learnable Parameters

When we use least squares to fit a linear basis function model: $y_i = \beta_0+\beta_1b_1(x_i) +\beta_2b_2(x_i)+\beta_3b_3(x_i)+...+\beta_kb_k(x_i)+\epsilon_i$ Is it possible to fit a model where the $...
0
votes
0answers
12 views

Should I calculate the real value of net exports using the PPI for a linear regression?

I am using real values for variables such as GDP and Effective exchange rate for my OLS regression I was wondering if I should use the Producer Price index to to find the real value of net exports or ...
2
votes
1answer
24 views

Variance of predicted value in a linear regression when $n \to \infty$

The following question is from Kutner's Applied Linear Statistical Models - Ch 2 - 2.12 To answer the question a few pieces of information are needed, provided below: What I gather the question is ...
1
vote
0answers
9 views

Intuitive explanation behind conditional mean independence?

From what I understand from $E(U|X) = 0$ is that: for every slice of X (i.e. if we fix X, we can get a (normal) distribution of Y values that tend (as the definition of normal distribution) to cluster ...
0
votes
0answers
16 views

Why should covariance of mean of Y and least-squares slope be 0?

I was studying simple linear regression analysis. I was studying the proof of variance of the least squares intercept estimator $\hat \beta_0$ which is in the image. I understand the proof, but can ...
0
votes
0answers
11 views

Interpretation of MAPE in Linear Regression model?

I made Linear Regression in Python and I have only 1 simple question. I have MAPE = 0.052 it means that my mean percentage error is 0.052% or 5,2% ?
1
vote
0answers
22 views

What do you mean by a model is adequate?

I'm learning about regression and stumbled upon this part where we check the accuracy of the model. In order to do that, we do an F test with the null hypothesis stating that the model equation is ...
0
votes
0answers
27 views

Least square for binary classification {0, 1}: is the threshold always equal to 0.5? [duplicate]

When using least square for classification of two classes labeled as 0 and 1, the value 0.5 is a common choice for the threshold. Intuitively, it makes sense, but is it always optimal? In which sense?
0
votes
0answers
11 views

Is a classification comparison between Linear Discriminant Analysis and a SVM trained on said linear discriminants appropriate?

I am currently trying to investigate the classification accuracy of two models on a wide dataset (79*222), with 4 balanced classes. The models are: Principal Component Analysis, Linear Discriminant ...
1
vote
0answers
8 views

Online Tree Based Algorithms

Linear regression and logistic regression can do online training(i.e. continuous training as new data arrives) via stochastic gradient descent. Are there any tree based algorithms which can ...
0
votes
1answer
25 views

Does this residual graph show bias?

A linear regression model generated the below residuals plot. How should we interpret this in terms of mean-zero error assumption? Is the assumption violated or not? Does this show bias?
2
votes
1answer
74 views

boundary (singular) fit: see ?isSingular

I'm running a linear fit model using the lme4 package on Rstudio and the following code keeps coming up with the error "boundary (singular) fit: see ?isSingular" can someone please help? m1= ...
0
votes
1answer
44 views

Which one is the regression line of x on y?

I have the two regression lines $6x+y=30$ and $3x+2y=25$. How do I identify which one is which? It is given that the first one is the regression line of x on y. How is that? Any ideas?
0
votes
1answer
16 views

Why is the expected value of the optimism of the training error for a linear model equal to $\frac{2}{ n}d\sigma ^{2}$?

In the book Elements of statistical learning 2 on page 229, they express the expected optimism of the training error as: $$ \omega=\frac{2}{N} \sum_{i=1}^{N} \operatorname{Cov}\left(\hat{y}_{i}, y_{i}\...
1
vote
0answers
56 views

Understanding Residuals vs Leverage plot in terms of meeting regression assumptions

Can someone help me understand the Residuals vs Leverage plot in terms of meeting the assumption of independence/influence for multiple linear regression models? My understanding is that the ...
1
vote
0answers
22 views

How to interpret a transformed linear regression model

I'm playing with the trafo R-package and this small data. After using the assumptions function, I found the log shift opt transformation is the best for normalizing ...
0
votes
0answers
26 views

How to explain features' importance after PCA?

I had a dataset that was terribly haunted by colinearity. I carried out PCA on it to remove the colinearity. Let's assume I am going to build a linear regression model, which is easily explainable, on ...
8
votes
1answer
155 views

How do Lawson and Hanson solve the unconstrained least squares problem?

For the non-negative least squares problem min(b - a %*% x) subject to x >= 0, the Lawson-Hanson fortran77 implementation (...
2
votes
4answers
79 views

Understanding the assumptions of Linear Regression

I have been studying the four assumptions of linear regression and different sources give different interpretations of the same. The four assumptions are : a) Linearity - Existence of linear ...
1
vote
0answers
25 views

Proof that variance-covariance matrix of var(b|X) - var(b*|X) is positive-semidefinite [closed]

I'm having trouble finding the proof to show that the variance-covariance matrix of var (b|X) - var (b*|X) is positive-semidefinite. OLS estimator = GLS estimator = Hint: Note that A is the Cholesky ...
0
votes
0answers
19 views

Are short and long regression coefficients the same when the conditional mean of regressand is independent of one regressor?

Consider three scalar random variable $Y,W,Z$ and the short linear regression $E[Y|W] = \alpha^*_{s} + \gamma^*_{s}W + U_s$ and the long regression $E[Y|W,Z] = \alpha^*_{l} + \gamma^*_{l}W + \delta^*_{...
2
votes
2answers
50 views

Do explanatory variables have to have a linear relationship with the response variables?

Do explanatory variables have to have a linear relationship with the response variable in multiple linear regression? What is the reason for this assumption? Also, why are heteroscedastic ...
2
votes
2answers
39 views

Formal statistics vs naive Statistics

In the last year I started to study Data Science by some Udemy and Coursera courses. As a pure mathematician, my curiosity makes me study statistics more formally and deeper. Yesterday, I rewatched ...
0
votes
0answers
10 views

How to estimate the non-linear contributions in a discreet-time system, but without system identification?

The question: how to estimate the nonlinear contributions in a non-linear discreet-time function with random error source without performing any actual system identification? As a part of a work ...
0
votes
0answers
15 views

How to show that prediction error estimated by least square is not correlated with any linear function of the training data?

Consider simple linear regression $\hat y = b \cdot x = \sum b_ix_i$ where $x$ is a vector. Let $b^*$ be the best possible linear regression parameters estimated from training data using least square ...
0
votes
0answers
7 views

Simultaneous linear and quadratic lm fit to different parts of the data

I have dependent variable x, indpendent variable y and group membership idx. There is a ...
0
votes
0answers
31 views

Covariance matrix as a linear transformation

I am trying to understand the general relationship between the covariance between two random variables and linear transformations. For example, consider the figure here: https://en.wikipedia.org/wiki/...
5
votes
1answer
93 views

How to find se.fit in R by hand?

I'm trying to find by hand the se.fit. Focusing on the first observation (weight=$1.9$), when we write the following code: ...
1
vote
3answers
49 views

Integrating an absolute difference transformation

I have a dependent variable which is $$ W = |X - Y| $$ X and Y are independently distributed where $ X \sim Uniform(0,1)$ and $ Y \sim Uniform(0,2)$ What am I supposed to do is find the probability of ...
0
votes
0answers
21 views

Magnitude of Error vs. Model Precision

In a simple linear regression model, is it safe to make a broad statement that the magnitude of the error is rooted in summing residuals and the precision of the model is rooted in the distribution of ...
1
vote
1answer
27 views

Compute the slope of the straight portion of the graph

I have the following four graphs for which I have to compute the slope of the straight portion of the graph (it returns the value of Young's Modulus of a material). I initially received a 4 datasets ...
2
votes
1answer
30 views

Intuitively Assert Consistency

Consider a model $$\mathbf y=\beta_0+\mathbf x_1\beta_1+\mathbf x_2\beta_2+\mathbf e$$ and assume $\mathbb E[\mathbf e\mid\mathbf X]=0$. Under this scenario, the OLS estimator is a consistent ...
0
votes
0answers
13 views

LDA QDA with constant value of 0 in dataframe

I have an issue with LDA and QDA methods. I noticed that when I use these methods on a dataframe full of dummy variables ( possibility of columns with only with the value of 0) , these methods don't ...
0
votes
0answers
24 views

Standardized regression coefficient vs Pearson's correlation coefficient?

When we approximate a general model (not necessarily linear) by a linear regression model (with intercept), I wonder why we should favor the Standardized Regression Coefficient against the Pearson ...
0
votes
1answer
20 views

How do I figure out the specific coefficient of a dummy variable?

I have a linear regression model that aims to predict the quality of melon icecream (dependent variable) using the amount of sugar, melon powder, and vegetable oil. The equation is as follows, $$\text{...
1
vote
1answer
17 views

Is linear regression possible to ascertain time is not a confounding factor on treatment outcome?

I am currently working on a small observational and retrospective study with 2 groups with different types of treatment. We have 3 outcome measures for both groups. The outcome measures were obtained ...

1
2 3 4 5
23