# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

2,588 questions
Filter by
Sorted by
Tagged with
8 views

### Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]

In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
• 561
13 views

### Performing deconfounding using multilinear regression for select number of variables

I'm trying to perform a correction on my data so I can use deconfounded residuals in later analyses. My data object is a subject by observation matrix (each subject has i observations - if it helps ...
• 23
111 views

### Am I interpretting this interaction term correctly?

I am running a model with an interaction term and I am unsure of the interpretation even after reading the other questions here in the forum. My model looks as follows: Where roi is the return on ...
19 views

### Prove that the sample covariance between observation and OLS fittings are nonnegative

I am trying to show that $$\frac{1}{n} \sum_{i = 1}^n (y_i - \overline{y})(\hat{y}_i - \overline{\hat{y}}) \geq 0$$ where $y_i$s are the observations, $\hat{y}_i$s are corresponding LS fitting values, ...
• 111
29 views

### Fixed effect model: different estimation approaches with R - how to demean variables - unbalanced panel

I want to use R to estimate a fixed effects model using different estimation approaches. Note that I am using an unbalanced panel. The easiest way to do this is using the function ...
37 views

### How can I compute correct standard errors after implementing the FWL Theorem?

I am trying to implement the FWL theorem for some sample data in Stata. This theorem tells us that given a multivariate regression of the form $y = \beta_{1}x_{1} + \beta_{2}x_{2} + \varepsilon$, the ...
12 views

### Are the assumptions and implications for ordinary least squares listed relevant, comprehensive or too-relaxed for generalized linear models? [closed]

The following diagrams show which assumptions are required to get which implications in the finite and asymptotic scenarios. I have no idea whether GLMs share these assumptions or whether GLMs have ...
184 views

28 views

### Least squares with equality constraint [migrated]

Say I have the following Least Squares equation with constraints and constant parameters $a_i$: $\min(\sum(x_{i}-a_{i})^2), \sum{x_{i}}=1,x_i>0$ Basically, I am looking for the best set of $x_i$'s ...
• 219
27 views

• 111
10 views

### How should I interpret the results of the OLS regression I did for 2 cointegrated variables?

So I've been doing cointegration between two variables that are both I(1). I run the OLS regression between the variables to possibly check the stationary of the residuals. However when I checked the ...
16 views

### A Seeming Discrepency in least-square (LS) Regression Analysis

I encountered a seeming discrepency in least-square (LS) regression analysis. The dependent variable is y and the independent variables are x1, x2, x3 ... If I perform a LS regression with these ...
1 vote
249 views

### Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
• 976
23 views

### Showing the unbiased estimator of variance for GLS estimator

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
• 976
31 views

### OLS solution to linear regression via SVD decomposition

I'm solving a linear regression problem. In a textbook that I follow, the author says that directly computing the OLS vector: $\beta = (X^TX)^{-1}X^T y$ can lead to problems when $(X^TX)$ is singular ...
• 111
50 views

### MLE to address multicollinearity in linear regression

OLS estimation assumes that the explanatory variables are independent in the linear regression model. There isn't such assumption when using the MLE estimation. So, my question is, can we use MLE to ...
• 11
1 vote
16 views

### How to interpret the coefficient of a limited independent Variable (Index)?

I assume this is a very simple question, however I am not sure about it. I have a regression table in front of me that contains the coefficients of a linear regression. The dependent variable is ...
• 95
15 views

### Dummy variable trap in OLS with multiple indicator variables

My dataset contains two numerical variables (n1, n2) and six indicator variables. The first three indicator variables specify ...
• 123
49 views

### Why is Ω unknown in GLS?

We run OLS and found the Homoscedasticity is violated and Hence, we go for GLS. But from variance-covariance of OLS's error - we have already found the Ω. Now, if we want to estimate β coefficients ...
• 153
66 views

### Help: Partitioned samples efficiency in OLS compared to one sample regression

As usual, we can estimate by OLS the model (in matrix form) $Y=\alpha+\beta*X+u$ with a sample of $n+m$ observations. The OLS estimator is $\hat{\beta}=(X^{T}X)^{-1}X^{T}Y$. Now, if we partition our ...
• 43
101 views

• 557
17 views

### Prediction with uncertainty after least-square estimation

I've fit a model (the solution to differential equations or some other non-linear functions) to observational data to estimate the best-fitted parameters and their uncertainty by least-square methods (...
• 11
57 views

### The Literature on the impact of outliers on ordinary least square (OLS) regression

I remembered I have encountered a paper in 1960s or 1970s that explore the impact of outliers on ordinary least square (OLS) regression. In the paper, it is shown that just adding one outlier will ...
12 views

### If linear regression is parametric, do we need normality of the features and/or target? [duplicate]

From what I know, linear regression is a parametric model (as mentioned in here). Parametric tests requires normality of the variables. My first question is that this is an assumption of the linear ...
44 views

### Linear regression model with a distribution over regression equations

Suppose that the observations $(y_t, x_t, k_t)_{t=1}^N$ satisfy the linear regression equation: \begin{equation} \begin{split} y_t = \begin{cases} x_t \beta + e_t & w.p. \; \theta \\ k_t \gamma + ...
• 103
17 views

### How to manually calculate the variance of the least squares estimator in R [closed]

As stated in the title, how do you manually calculate the variance of the least squares estimator in R? I know that the least estimates have the following formula: $$\hat{\beta}=(X^TX)^{-1} X^T Y,$$ ...
• 147
1 vote
15 views

### Independent variables affected by size, how I resolve this problem?

I am doing a research about the impact of trade unions on permanent contracts. The dependent variable is trade union presence (0,1), the independent variable is number of permanent contracts ...
314 views

### Log of a log-transformed variable

I have been suggested to use the log of a log-transformed independent variable (i.e., log(log healthcare expenditure)). I am not sure how would this make sense. Is this a standard practice (in the ...
• 61
1 vote
37 views

### How can I estimate the sum of coefficients

I am trying to estimate the cumulative effect. When I have an ols regression with many dummies as explanatory variables, can I sum the coefficients to find the cumulative effect? If yes, how do I find ...
• 11
7 views

### Confidence bounds for coefficients of a fit of data set obtained with another fit

I fitted an equation to a set of data points. Then I substracted the fit previously obtained to another set of data points. After that, I fitted another equation to this new data (result of the ...
1 vote
13 views

25 views

### Confidence intervals for non-negative least squares

Can we use the non-parametric bootstrapping to compute the confidence intervals for the regression coefficients estimated from non-negative least squares? I wonder whether this has the same issues ...
• 601
1 vote
29 views

### How to interpret an index of values between -2.5 and +2.5 (an independent variable) in a regression?

I am in the proccess of writing my Master's Thesis and I'm performing a multivariate regression (OLS). One of my independent variables is Chinn-Ito Index (financial openness index) which takes values ...
• 11
1 vote
Assuming the true equation for Y is linear as below: $$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$ Assuming X is fixed, then the variance of each Y is: $$var(Y_i )=var(\epsilon_i)=\sigma^2$$ In order to ...