Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]

In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
0 votes
0 answers
13 views

Performing deconfounding using multilinear regression for select number of variables

I'm trying to perform a correction on my data so I can use deconfounded residuals in later analyses. My data object is a subject by observation matrix (each subject has i observations - if it helps ...
  • 23
2 votes
2 answers
111 views

Am I interpretting this interaction term correctly?

I am running a model with an interaction term and I am unsure of the interpretation even after reading the other questions here in the forum. My model looks as follows: Where roi is the return on ...
0 votes
0 answers
19 views

Prove that the sample covariance between observation and OLS fittings are nonnegative

I am trying to show that $$\frac{1}{n} \sum_{i = 1}^n (y_i - \overline{y})(\hat{y}_i - \overline{\hat{y}}) \geq 0$$ where $y_i$s are the observations, $\hat{y}_i$s are corresponding LS fitting values, ...
  • 111
0 votes
1 answer
29 views

Fixed effect model: different estimation approaches with R - how to demean variables - unbalanced panel

I want to use R to estimate a fixed effects model using different estimation approaches. Note that I am using an unbalanced panel. The easiest way to do this is using the function ...
  • 1
3 votes
1 answer
37 views

How can I compute correct standard errors after implementing the FWL Theorem?

I am trying to implement the FWL theorem for some sample data in Stata. This theorem tells us that given a multivariate regression of the form $y = \beta_{1}x_{1} + \beta_{2}x_{2} + \varepsilon$, the ...
-1 votes
0 answers
12 views

Are the assumptions and implications for ordinary least squares listed relevant, comprehensive or too-relaxed for generalized linear models? [closed]

The following diagrams show which assumptions are required to get which implications in the finite and asymptotic scenarios. I have no idea whether GLMs share these assumptions or whether GLMs have ...
6 votes
2 answers
184 views

Coefficient of an Interaction Term when Regressors are Independent

Say we have an OLS regression of $y$ on $x_1$ and $x_2$, where both $x_1$ and $x_2$ are independent from each other, and create the following regression: $$ y= \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \...
0 votes
1 answer
30 views

Annihilator matrix transforming all variables in deviations from their sample means?

In my econometrics notes (not a proper textbook) I find that given a partitioning of the sample matrix X into $$ X =(X_1 1),$$ where 1 is the (nx1) vector of all unity elements, then $$M_{[1]}=I - 1(1'...
0 votes
0 answers
28 views

Least squares with equality constraint [migrated]

Say I have the following Least Squares equation with constraints and constant parameters $a_i$: $\min(\sum(x_{i}-a_{i})^2), \sum{x_{i}}=1,x_i>0$ Basically, I am looking for the best set of $x_i$'s ...
2 votes
1 answer
27 views

Coefficient from Regressing the OLS Residual on X

Say we have an OLS residual $\hat{ϵ}$ from regressing $y$ on $X$. If we were to regress $\hat{ϵ}$ on the same $X$, what would the OLS coefficient be? If we rearrange the first regression equation for $...
's user avatar
0 votes
0 answers
7 views

MLR OLS computing slope and intercept separately

In multiple linear regression, when we want to derive the slopes and intercept separately, I have seen the following formulas: $\hat \beta = (X^T_c X_c)^{-1} X^T_c y_c$ $\beta_0 = \overline y - \beta^...
  • 111
0 votes
0 answers
10 views

How should I interpret the results of the OLS regression I did for 2 cointegrated variables?

So I've been doing cointegration between two variables that are both I(1). I run the OLS regression between the variables to possibly check the stationary of the residuals. However when I checked the ...
  • 1
0 votes
0 answers
16 views

A Seeming Discrepency in least-square (LS) Regression Analysis

I encountered a seeming discrepency in least-square (LS) regression analysis. The dependent variable is y and the independent variables are x1, x2, x3 ... If I perform a LS regression with these ...
1 vote
2 answers
249 views

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
0 votes
0 answers
23 views

Showing the unbiased estimator of variance for GLS estimator

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
2 votes
0 answers
31 views

OLS solution to linear regression via SVD decomposition

I'm solving a linear regression problem. In a textbook that I follow, the author says that directly computing the OLS vector: $\beta = (X^TX)^{-1}X^T y$ can lead to problems when $(X^TX)$ is singular ...
  • 111
0 votes
2 answers
50 views

MLE to address multicollinearity in linear regression

OLS estimation assumes that the explanatory variables are independent in the linear regression model. There isn't such assumption when using the MLE estimation. So, my question is, can we use MLE to ...
  • 11
1 vote
1 answer
16 views

How to interpret the coefficient of a limited independent Variable (Index)?

I assume this is a very simple question, however I am not sure about it. I have a regression table in front of me that contains the coefficients of a linear regression. The dependent variable is ...
  • 95
2 votes
1 answer
15 views

Dummy variable trap in OLS with multiple indicator variables

My dataset contains two numerical variables (n1, n2) and six indicator variables. The first three indicator variables specify ...
  • 123
0 votes
1 answer
49 views

Why is Ω unknown in GLS?

We run OLS and found the Homoscedasticity is violated and Hence, we go for GLS. But from variance-covariance of OLS's error - we have already found the Ω. Now, if we want to estimate β coefficients ...
  • 153
4 votes
2 answers
66 views

Help: Partitioned samples efficiency in OLS compared to one sample regression

As usual, we can estimate by OLS the model (in matrix form) $Y=\alpha+\beta*X+u$ with a sample of $n+m$ observations. The OLS estimator is $\hat{\beta}=(X^{T}X)^{-1}X^{T}Y$. Now, if we partition our ...
  • 43
5 votes
1 answer
101 views

Does Frisch–Waugh–Lovell theorem work on panel data?

We consider the following panel regression model \begin{align} Y_{i} = X_{1,i}\beta_1 + X_{2,i}\beta_2 + \epsilon_{i} , \ i=1,...,N, \end{align} where $Y_i := [y_{i1},...,y_{iT}]'$, $X_{k,i}:= [x_{k,...
  • 93
0 votes
0 answers
7 views

Interpreting the time series linear regression - differences before and after collapsing data

Consider the following time series: The coefficient on the linear regression makes sense: each additional year, the variable Y increases by 3 percentage points. Now, the problem occurs when I'm ...
0 votes
0 answers
12 views

Weighted Least Squares -- zero mean for error makes sense, but why require zero mean for measurements?

In Introduction to Linear Algebra, Strang shows \hat{x} from equation 12 in the attached screenshot is the best linear unbiased estimator for Ax = b. One of the assumptions used to derive \hat{x} is ...
0 votes
0 answers
17 views

OLS interpretation – if x decreases

Suppose I have a multivariate linear regression where the coefficient on Beta1 equals 0.05. All independent variables as well as the dependent variable are in percent. So the normal interpretation is: ...
2 votes
1 answer
44 views

Linear regression with convex combination of the parameters

I am looking for a method to solve the following linear regression problem: $$ y_i=\sum_{j=1}^Kx_{ij}\beta_j+\varepsilon_i $$ with all $\beta_j\geq0$ and $\sum \beta_j=1$. I am familiar with ...
2 votes
1 answer
48 views

How to establish relationship between regressions on subsets of data?

From classical OLS, the regression of $y\in\mathbb{R}^n$ on $X\in\mathbb{R}^{n\times k}$ yields $\beta = (X^TX)^{-1} X^Ty$. Suppose we were to partition $X$ into two blocks as: $X = \begin{pmatrix} ...
  • 557
0 votes
0 answers
17 views

Prediction with uncertainty after least-square estimation

I've fit a model (the solution to differential equations or some other non-linear functions) to observational data to estimate the best-fitted parameters and their uncertainty by least-square methods (...
  • 11
0 votes
0 answers
57 views

The Literature on the impact of outliers on ordinary least square (OLS) regression

I remembered I have encountered a paper in 1960s or 1970s that explore the impact of outliers on ordinary least square (OLS) regression. In the paper, it is shown that just adding one outlier will ...
0 votes
0 answers
12 views

If linear regression is parametric, do we need normality of the features and/or target? [duplicate]

From what I know, linear regression is a parametric model (as mentioned in here). Parametric tests requires normality of the variables. My first question is that this is an assumption of the linear ...
3 votes
1 answer
44 views

Linear regression model with a distribution over regression equations

Suppose that the observations $(y_t, x_t, k_t)_{t=1}^N$ satisfy the linear regression equation: \begin{equation} \begin{split} y_t = \begin{cases} x_t \beta + e_t & w.p. \; \theta \\ k_t \gamma + ...
  • 103
2 votes
0 answers
17 views

How to manually calculate the variance of the least squares estimator in R [closed]

As stated in the title, how do you manually calculate the variance of the least squares estimator in R? I know that the least estimates have the following formula: $$\hat{\beta}=(X^TX)^{-1} X^T Y, $$ ...
  • 147
1 vote
0 answers
15 views

Independent variables affected by size, how I resolve this problem?

I am doing a research about the impact of trade unions on permanent contracts. The dependent variable is trade union presence (0,1), the independent variable is number of permanent contracts ...
5 votes
1 answer
314 views

Log of a log-transformed variable

I have been suggested to use the log of a log-transformed independent variable (i.e., log(log healthcare expenditure)). I am not sure how would this make sense. Is this a standard practice (in the ...
  • 61
1 vote
0 answers
37 views

How can I estimate the sum of coefficients

I am trying to estimate the cumulative effect. When I have an ols regression with many dummies as explanatory variables, can I sum the coefficients to find the cumulative effect? If yes, how do I find ...
  • 11
0 votes
0 answers
7 views

Confidence bounds for coefficients of a fit of data set obtained with another fit

I fitted an equation to a set of data points. Then I substracted the fit previously obtained to another set of data points. After that, I fitted another equation to this new data (result of the ...
1 vote
0 answers
13 views

Geometric Interpretation of GLS with endowment of new norm

I was reading a very short passage about GLS (Generalized Least Squares Regression) provided with insufficient reference. I understand the derivation process of the BLUE $\hat{\beta_G} = (X^TV^{-1}X)^{...
0 votes
0 answers
26 views

OLS After LASSO to remove negative coefficients

I have a regression model with many predictors and not that many instances. (~70 predictors and 150 instances) I would like to use the model for inference, and therefore need to identify the sparse ...
  • 203
1 vote
0 answers
43 views

Potential problem with multicollinearity?

I am trying to figure out what happens to my results if I unintentionally introduced multicollinearity. I have an unadjusted version of this regression that includes an interaction term. The ...
0 votes
0 answers
26 views

Proc Mixed LSM standard error doesn't equal manual calculations

My experiment has 3 raw samples, and 17 derived products, made with different processing conditions. The raw samples were done in triplicate, where as the derived products were made in duplicate (i.e.,...
1 vote
0 answers
24 views

Does including an irrelevant explanatory variable have more consequences for OLS estimation results than the omission of an explanatory variable [closed]

“The inclusion of an irrelevant explanatory variable in a regression has more serious consequences for OLS estimation results than the unjustified omission of an explanatory variable.” Explain whether ...
  • 11
0 votes
0 answers
77 views

What can OLS with a Box-Cox transformed dependent variable tell me?

Just to ellaborate: I’m doing an OLS-test to determine the following things: Do my independent variables have a significant effect on the dependent variable? What’s the direction of the effect of my ...
  • 1
3 votes
1 answer
83 views

What is the relationship between linear regression and z score regression?

So I'm taking a stat's class that has introduced z-score regression. According to my professor, z-score regression gives us the "line of best fit" when the data has a linear structure. I've ...
  • 131
0 votes
0 answers
32 views

Composite Null hypothesis in a Linear Regression

Suppose that we have an OLS estimator $\hat{\beta}$. Also, assume that we know $\hat{\beta}\sim N(\beta, \Omega)$ and have an estimator for $\Omega$, say $\hat{\Omega}$. Here, I want to test the ...
  • 713
1 vote
1 answer
23 views

standard error derivation using FWL theorem: Is this correct?

I'm reading a paper (https://scholar.google.com.br/scholar?oi=bibs&cluster=986729284887040990&btnI=1&hl=en), and there's this derivation in the article which I'm not sure is correct, so I ...
  • 21
3 votes
1 answer
26 views

Accuracy of coefficients on log-linear regressiosn

Consider the following two regression models using data from the table below. $ln(wage)=\beta_0+\beta_1 female+u$ $ln(wage)=\gamma _0+\gamma_1 male+v$ wage female male 10 1 0 20 1 0 30 1 0 40 1 ...
0 votes
0 answers
25 views

Confidence intervals for non-negative least squares

Can we use the non-parametric bootstrapping to compute the confidence intervals for the regression coefficients estimated from non-negative least squares? I wonder whether this has the same issues ...
  • 601
1 vote
1 answer
29 views

How to interpret an index of values between -2.5 and +2.5 (an independent variable) in a regression?

I am in the proccess of writing my Master's Thesis and I'm performing a multivariate regression (OLS). One of my independent variables is Chinn-Ito Index (financial openness index) which takes values ...
1 vote
1 answer
50 views

why not using sample variance (instead of MSE) to estimate the error variance in linear regression?

Assuming the true equation for Y is linear as below: $$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$ Assuming X is fixed, then the variance of each Y is: $$var(Y_i )=var(\epsilon_i)=\sigma^2$$ In order to ...
  • 179

1
2 3 4 5
52