# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

2,085 questions
Filter by
Sorted by
Tagged with
11 views

### PRESS from the hat matrix and numerical stability from statsmodels ols.fit()

Leave one out cross validation in the context of ordinary least squares regression can be done via the hat matrix: The "hat" or projection matrix $$H = X(X^T X)^{-1} X^T$$ many fit ...
22 views

### Relation between OLS, MM and ML

What is the relation between OLS, MM (method of moments) and ML (maximum likelihood)? During my studies, the three concepts got taught completely separated from each other. However, they seem to be ...
30 views

### In OLS regression, when the assumption of normally distributed residuals is rejected, is bootstrap (and block bootstrap) the way to deal with it?

In OLS regression, when the assumption of normally distributed residuals is rejected, is bootstrap (and block bootstrap) the way to deal with it? Is this the right way to go or non-normally ...
8 views

### Curious results with t statistic with GARCH errors in a linear regression

So, I was playing around with an odd specification in a simulation experiment: \begin{align} y_t &= x_t \beta + \sqrt{h_t} \epsilon_t \\ h_t &= \sigma^2 + \pi \left( h_{t-1} - \sigma^2 \...
9 views

I have 1000 data points from the bivariate normal distribution $\mathcal{N}$ with mean $(0,0)$ and variance $\sigma_1^2=\sigma_2^2=10$ with the covariances being $0$. Also there are 20 more points ...
30 views

### Square loss for “big data”

Let’s set up a supervised learning problem with $p$ predictors and $n$ observations. The response variable is univariate. The problem can be regression or classification, though I think a ...
63 views

### If an OLS model is estimated without an intercept (no constant term) but the average residual is close to zero, are we OK?

Let's suppose an OLS model is estimated without an intercept (no constant term), but the mean residual is very close to zero (2.2E-11) does that mean the model is OK to have been estimated without a ...
23 views

19 views

### Subtracting a constant from the OLS summation

On a proof for the OLS of $\beta$, I have seen this step: $\sum x_i (y_i - \alpha - \beta x_i) = \sum (x_i - k) (y_i - \alpha - \beta x_i)$ for any constant $k$. Why is this true?
38 views

### What is the difference between least squares line and the regression line?

It is common to plot the line of best fit on a scatter plot when there is a linear association between two variables. One method of doing this is with the line of best fit found using the least-...
22 views

### Multiply beta coefficients from two different models

I have a count outcome with a heavy right skew that is modeled with a negative binomial. I have a continuous mediator that is modeled with OLS. We're attempting a method of causal mediation analyses ...
17 views

### Existence of least squares and maximum likelihood estimators?

In statistical parameter estimation where there is a deterministic and stochastic component to the observation-generating model, do least squares and maximum likelihood estimators always exist? ...
12 views

### The OLS and CSS deterministic trend with AutoRegressive Model

I use two method to get the coefficient of AR(1)+trend Model, one of the method is "CSS" and another "OLS". First i generate a time series ar(1) with trend and mean = 10. ...
12 views

### How do you check whether heteroscedastic-consistent estimators fix your error variance problem?

I am running an OLS regression, and have non-constant error variance (residuals vs fitted looks like a fan opening up to the right). I have tried a number of power transformation but they seem to make ...
50 views

### Interpreting Estimated Coefficients of Linear Regression

I have data that requires interpretation of the below coefficients. Description of variables: "region" = the beneficiary’s residential area in the US; a ...
10 views

### In multivariate regression, under what condtions is $var(X_i\epsilon_i')$ positive definite?

Suppose we have $(Y_i, X_i)$, with $Y_i$ an r.v. in $\mathbb{R}^k$ and $X_i$ an r.v. in $\mathbb{R}^p$ and suppose the covariance matrix of $X_i$, $E(XX')$ is positive definite. Now we can estimate ...
32 views