# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

598 questions
585 views

### Bound for Arithmetic Harmonic mean inequality for matrices?

NOTE: This question has originally been posted in MSE, but it did not generate any interest. It was first posted there, because the question itself is a pure matrix-algebra question. Nevertheless, ...
168 views

98 views

### Dealing with autocorrelation using Generalized Least Squares

I have a time series data set where the auto correlation of the residuals follow an exponential decay. I was wondering how I should deal with this? I would like to fit a linear model and have tried ...
47 views

### Kernel and regularization parameter of James–Stein estimator

Consider a FIR model of the form $y= Ug_0+e$ with $e$ white noise with variance $\sigma^2$. We assume that we have collected N input-output measurements $y$ and $U$. The James–Stein estimator is ...
39 views

### Intepretting Linear Mixed Effects Model Output

I'm trying to reconcile the output of this MixedLM Model Output with my knowledge of OLS model outputs, e.g., MixedLM has Z-Stats vs. T-Stats for OLS. Model code for reference, but this is not my ...
40 views

### Direction of Bias in OLS with systematic measurement error

I am looking at a study that wants to measure the effect of $x_t$ on $Y_t$, but the true values of the $x_t$ are not observed. Instead, what is observed is a minimum value for $x_t$. In effect, what ...
96 views

### Can “non-parametric” tests be achieved with generalized linear models?

I recently read @Kodiologist's answer to a post here looking for clarification on the relation between GLMs and non-parametric tests. His answer is along the lines of "the approach is not non-...
69 views

### small sample approach to simple linear regression with errors-in-variables (measurement errors)

I seek to estimate $b_1$ and $b_0$ from data of the form: $$y_i = b_1x_i + b_0 + e_i, \quad i\in\{0,1,...,N-1\}$$ given $\{y_i\}$ and $\{\tilde{x}_i\}$ where $\tilde{x}_i=x_i + n_i$ (i.e., error-in-...
657 views

### Why use quantile regression instead of splitting the data in quantiles and calculating multiple linear regressions?

Why use quantile regression instead of splitting the data in quantiles and calculating multiple linear regressions? What are the advantages and disadvantages of these methods? As far as I understand ...
88 views

28 views

### Loess preprocessing for robust quadratic regression?

It seems to me that we could fit a quadratic curve to the predictions of a LOESS curve in order to obtain a parametric model approximating the non-parametric LOESS model. Is this something that is ...
### LASSO solution when $p \gg n$
We know that when $p \gg n$ we don't have a unique least squares solution. But, in the same scenario, does LASSO have a solution? LASSO uses the least squares term as part of its cost function (i.e.,...