Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

974 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3k views

Fitting a Logistic Regression via Brier Score or Mean Squared Error

Is there a name for a logistic regression model that has been fit using the Brier score (or equivalently the mean-squared error) rather than the cross-entropy? I realise this isn't maximum-likelihood, ...
• 55.1k
195 views

• 1,118
302 views

Simultaneity of price in sales modeling

The price of a product has signficant impact on the total sales. Hence modeling sales would give the incentive to include price as a regressor (amongst other variables). Suppose we would estimate this ...
679 views

Least squares robust to outliers

I have sparse overdefined system of linear equations. For example I have n variables, m equations(m>n) and k equations from m are "bad" equations that represent outliers. Is there any methods to ...
• 2,217
112 views

Kalman Filter to minimize weighted errors on the states: what's wrong with my derivation

I am thinking about how to implement a "weighted Kalman Filter". Note that the weights here are on the states. Basically the classical KF minimizes $\sum (x_i - \hat{x_i} )^2$ but I want to ...
• 230
34 views

Is there a transformation that could inverse the residuals in multiple OLS regression?

Let's say I have a partial residual plot that looks like this, where the residuals are predicted minus actuals. I would instead prefer for the residuals to be inversed, so that instead of ...
• 3,253
73 views

OLS estimator and conditional variance weighting

I'm reading Counterfactuals and Causal Inference by Morgan and Winship. In chapter 6, they discuss OLS as a means of estimating the average treatment effect for a binary exposure $D$ (assuming all ...
• 38.6k
33 views

• 435
68 views

Proof that p-value in OLS regression is symmetric

OLS regression is not symmetric, meaning that it produces different relationships if you flip the dependent and independent variables; however, it would seem odd if the p-values were different and ...
• 283
1k views

Can I interpret control variable's coefficients in linear regression?

imagine I want to estimate a regression model (let's say OLS for simplicity's sake) using observational data. I include a number of controls that might confound the relationship. For example, I might ...
177 views

Why orthogonal polynomials lead to diagonal matrix $X^{T}X$ when estimating regression estimates?

We believe that the response variable $Y$ can be modeled in a following way: $$Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 + \epsilon_i$$ where $\epsilon_i$ is independently ...
• 604
115 views

Estimating a linear system of simultaneous equations at once

Consider the following simultaneous system $$y_{1} =\beta _{1}y_{2}+\alpha _{1}z+u_{1} \\ y_{2} =\beta _{2}y_{1}+\alpha _{2}z+u_{2}$$ where $y_{1}$, $y_{2}$ and $z$ are vectors of random variables ...
249 views

Estimating the errors in parameters in the ordinary least square

I am reading the book An Introduction to Error Analysis by John R. Taylor. In Ch8: Least-Squares Fitting, he has derived expressions for parameters $A$ and $B$ in fitting the line $A+Bx$ to the set of ...
• 623
29 views

terms in a simple linear least square model

I'm reading a textbook. In the chapter about least square regression I red that A simple linear least square model can be described as $$Y = \alpha + \beta x + e$$ where Y ...
• 409
451 views

I'm conducting a multiple OLS regression. My main model contains a significant effect (p < .5) of x on y. I want to test in a robustness check whether x is related to y in a curvilinear/quadratic ...
• 647
679 views

What is the effect of autocorrelation on R² in OLS

I was wondering. What would be the effect of the presence of autocorrelation on R² or R²_adjusted? Especially in a dataset used in multiple linear regression? edit: So the correlation of each variable ...
77 views

Why is my QR decomposition updating code numerically off?

I apologize if this is the wrong place for this question; there are a number of potential points of failure each of which suggest either Math StackExchange or StackOverflow or here, but since the ...
• 1,373