# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

2,355 questions
Filter by
Sorted by
Tagged with
26 views

### Regressor contribution in OLS regression

Assume I have the following model, estimated using OLS: $Y_{it}=β0+β1∗X1_{it}+β2∗X2_{it}+β3∗X3_{it}+ϵ_{it}$ I know that some methods exist to compute the relative contribution of each variable to the ...
66 views

76 views

35 views

### Understanding difference between Maximum Likelihood and Levenberg Marquardt result

In some of my regression results I noticed a deviation between Maximum Likelihood (via Monte Carlo Markov Chain, initialised by parameter result of Nelder-Mead, median value pictured) result and ...
44 views

### How do we interpret the "natural log of income"

I'm just confused as to how we interpret a natural log in relation to a regressor, say years of training thanks
41 views

### What is scikit-learn's LinearRegression doing when there are more features than observations? [duplicate]

I'm trying to understand what sklearn's LinearRegression (which should be using ordinary least squares) is doing when there are more features than observations. ...
18 views

### What are the calculations or maths behind least-squares-minimizing in linear regression used by sklearn [duplicate]

I'm relatively new in the ML field, and this question came up when working with linear regression from sklearn library. After a bit of looking up in the documentation, it states Compute least-squares ...
77 views

### What if the prior not be conjugate with posterior in Bayesian learning?

I know that when the prior is conjugate with posterior then one can get an analytical representation for the posterior distribution, but what if these two are not to be conjugate? For example, I would ...
28 views

### OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
35 views

### Unit of measure in the OLS regression

Suppose I estimate by OLS the following linear model $$Y_i=\beta_0+\beta_1 X_i+U_i$$ where $Y_i$ denotes the weight of individual $i$ (in pounds), and $X_i$ denotes the height of individual $i$ (in ...
24 views

15 views

### Cross-validation for Ordinary least squares

I want to do a ordinary least-square (OLS) fit to points (no error bars, they are not measurements) to find linear coefficients. I know that the model is imperfect, and want to quantify the model ...
10 views

### Change in coefficient β in log linear model

I'm studying econometrics with Wooldridge's manual. In a problem from Chapter 2, you have to prove that the coefficient β1 does not change when you transform a simple linear regression into a log-...
103 views

### Fitting a Logistic Regression via Brier Score or Mean Squared Error

Is there a name for a logistic regression model that has been fit using the Brier score (or equivalently the mean-squared error) rather than the cross-entropy? I realise this isn't maximum-likelihood, ...
35 views

19 views

### minimize sum of MSEs under orthogonality constraint

Suppose we want to minimize the objective $L=\frac{1}{2}||y-X\beta_1||^2 + \frac{1}{2}||y-X\beta_2||^2$ over $\beta_1, \beta_2$, under the constraint $\beta_1^T\beta_2=0$. Is this equivalent to first ...
30 views

### Where does this formula for the OLS estimate as a function of the true parameters come from?

Suppose that in the population: $$y = \alpha + \beta_1 x_1 + \epsilon$$ We now estimate the model: $$\hat{y} = \hat{\alpha} + \hat{\beta_1} x_1$$ I have seen the following formula which writes the ...
10 views

### Writing OLS estimates as a function of real parameters in case of multivariate regression

Suppose that in the population: $$y = \alpha + \beta_1 x_1 + \epsilon$$ We now estimate the model: $$\hat{y} = \hat{\alpha} + \hat{\beta_1} x_1$$ If we try to estimate $\beta$ using OLS, we have ...
48 views

### Differences between least-squares estimate and weighted least-squares estimate when I focus on only one coefficient in a multiple linear regression?

Suppose we have $n$ samples from the following linear model: $$y=x_1 \beta_1+x_2 \beta_2 + e,$$ where $e_i$ i.i.d. comes from $N(0,\sigma^2)$; $x_1$, $x_2$ are centralized with mean $0$. I am only ...
42 views

### If a fitted OLS regression model is mis-specified, is it possible to produce a second model that is unbiased?

Let’s say I want to build a linear regression model to conduct some sort of statistical inference. I plan on using the least squares method to fit the model. My understanding is that you need to ...
15 views

### understanding R2 in probit

I try to create a model to predict football (socker) results with a performance variable. It doesn't really matter how this performance is calculated since any performance variable is an adequote ...
My question is based on this question. Suppose we assume the sample is iid (so time series data is out) and $E[e_i X_i ] = 0$ but we're not sure about $E[e_i \mid X_i]$ = 0. Can you provide a ...