# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

2,868 questions
Filter by
Sorted by
Tagged with
44 views

### Linear model between data with standard error

I have two set of data from a experiment. I would like to evaluate a linear model between both sets. However, the term corresponding to y variable is composed by two elements: The measure and its ...
1 vote
51 views

### Is Fisher's discriminant analysis equivalent to the Bayes optimal LDA when the no. of classes is greater than two and covariances are all equal?

P.S. While I gave a brief background to make the question complete, informed readers can move to the questions 1 and 2 towards the end of this post, right after 'what are not clear to me are:'. Fisher'...
• 761
50 views

### Diff in Diff: wrong control group or wrong method?

I want to identify the causal effect of renewable energy targets on the environmental policy stringency index (I got it from OECD) for EU countries. My hypothesis is that by setting a renewable energy ...
1 vote
34 views

### Decision boundary for Cross entropy loss and Least square loss

We can see the source in this paper. My question is that why cross entropy loss has a boundary line in slope but least square loss has horizontal boundary. Can somebody explain?
• 483
44 views

### Ordered Probit with 20+ categories

Is there any upper limit on categories I can use for the dependent variable in an ordered probit model? In my current model I have at least 20 categories, but I maybe require more (up to 50). Is this ...
1 vote
80 views

### Unbiased and constistency of OLS

For the linear regression $y_t = Bx_t+e_t$ where we have the assumptions: $E(e_t)=0$, $E(e_t^2) = \sigma^2$, $E(e_t e_s)= 0$ for $s\neq t$ ...
• 21
1 vote
53 views

### How can I test if the OLS slope is different from a expected slope? [duplicate]

I am currently testing if an OLS slope is statistically different from a known, expected slope. I think I would need to do a one-sampled t-test but I am not exactly sure how. Specifically, is the ...
• 11
40 views

### Calculus versus matrix representation in OLS

In the Wikipedia article Ordinary Linear Squares there is an example for finding the estimators $\beta_i$ for a linear model of the sort: $$y_i = \beta_0 + x_1\beta_1 + x_2\beta_2 + \ldots$$ In the ...
• 135
54 views

### Sampling distribution of ordinary least squares confusion

I was reviewing the derivation for the variance of ordinary least squares estimators and experienced some confusion. $$\Large Var(\hat{\beta}) = \frac{\sigma^2}{\Sigma^n_{i=1}(X_i-\bar{X})^2}$$ From ...
11 views

### What's the point of using gradient descent for linear regression if you can calculate the coefficients directly using the least squares method? [duplicate]

Gradient descent involves significant computational effort, whereas the method of least squares enables direct and accurate calculation. Does gradient descent offer any advantages over least squares ...
20 views

• 2,966
60 views

### Classical linear regression with no independent sample

I like to organize my studies always with the weakest hypotheses possible. In this case, I want to understand well what assumptions I should add to be able to study linear regression models in time ...
133 views

### Estimating exponent of Zipf distribution using MLE vs fitting linear regression on log-transformed rank and frequency data

I'm having trouble understanding why I get radically different results if I try to find the parameter of a Zipf distribution when I use the methods proposed by Clauset et al. (2009) as opposed to ...
71 views

### How to determine if my model is robust? Should the coefficients be same?

I want to run robustness tests for my model. For example, by reducing the sample to heavily concentrated groups, running a different regression (probit etc) etc. But, how do I ascertain that my ...
• 245
1 vote
18 views

### Does min-norm least squares solve regular least squares in some basis?

For a data matrix $X$ of dimension $n \times p$ where $p > n$ and corresponding label vector $y$ of dimension $n$, the standard least squares fit, $\hat{\beta} = (X^TX)^{-1}X^Ty$, is ...
• 556
1 vote
96 views

### OLS with $iid$ Cauchy errors still unbiased?

A comment to this question suggests that the OLS estimate of linear model parameters is unbiased, even when the error term is Cauchy. Given that Cauchy distributions lack an expected value, I am ...
• 65k