# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1,586 questions
12 views

### Measure of error for non-linear systems?

I am trying to model a system that is ideally explained by the formula Q=Qi*(1+Di*b*t)^(-1/b), where: ...
18 views

### Confidence interval around weighted sum of regression coefficient estimates?

Let's say you have $N$ random variables $Y_i$, where $Y_i = \beta_i X + \epsilon_i$. $X$ values are the same for all $Y_i$, but the error terms have different variance. I estimate each $\beta_i$ with ...
42 views

### divergence of beta estimates between OLS and regression with ARIMA error

I have physiological time-series data: ~60k observations per channel, ~100 Hz sampling. I will model individual channels with ~20 regressors. Under OLS, given temporal autocorrelation in the data, ...
13 views

### confusion with frisch lovell type situation

Suppose I am trying to run a regression as follows: $y= a_1+ b_1*x_1 +b_2*x_2 + e_1$ To study the relation between $y$ and $x_2$ net of their correlation with $x_1$, I run the following model first: ...
34 views

### How can I find the k-nearest neighbors for a collection of linear time series data?

I need to figure out how to determine the nearest neighbors of an "optimal" line, as illustrated in a simplified figure, linked below. The blue, orange, green, and purple lines represent the best fit ...
30 views

### Interpret orthogonal design

I am dealing with orthogonal matrices in regressions, so every regressor has $x_j'x_k=0, if \ j \neq k \\ x_j'x_k=1, if \ \ j=k$ The first one told us that the correlation between two predictors ...
70 views

### Ridge/Lasso for correlated response

I want to try a penalised linear regression (ridge/lasso) as a comparison to standard OLS for its predictive ability. My response variable is a continuous measure of an eye parameter, so there is (...
28 views

726 views

### Is there any theoretical problem with averaging regression coefficients to build a model?

I want to build a regression model that is an average of multiple OLS models, each based on a subset of the full data. The idea behind this is based on this paper. I create k folds and build k OLS ...
26 views

73 views

### Converting the beta coefficient from matrix to scalar notation in OLS regression

I've found for my econometrics exams that if I forget the scalar notation, I can often save myself by remembering the matrix notation and working backwards. However, the following confused me. Given ...
33 views

### OLS and Probit possible on large sample enough?

I think I understood that normality of residuals may not be a problem if the sample is large enough (cf, here). My question is: Would my sample be large enough to be analysed using a probit and an ...
48 views

### Sum of squared residuals and MSE

It seems that minimizing the sum of squared residuals (SSR) in linear regression is equivalent to minimizing MSE (both use true value - prediction) and OLS is the best estimator for minimizing SSR. I ...