Skip to main content

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

Filter by
Sorted by
Tagged with
0 votes
2 answers
25 views

Different Transformation of the same IV [closed]

Suppose I've a panel data with 2 segments. And I want to run pooled OLS regression. I'm doing data transformation for each segment separately. Say for segment A, I'm doing log transformation and for ...
Beta's user avatar
  • 6,386
0 votes
0 answers
23 views

Regression with noises in X. Should I use the unbiased estimator or the OLS estimator for forecasting?

I am working with a dataset that includes variables $Y$ and $X$. I assume that $$ Y = \beta X + \epsilon $$ satisfies all the assumptions of OLS. Based on industry knowledge, I know that theoretically ...
The One's user avatar
  • 225
1 vote
1 answer
57 views

OLS vs MLE when errors are not normally distributed (Laplace distributed)

We say that under assumptions of the Gauss-Markov theorem, OLS is BLUE. The Gauss-Markov theorem doesn't mention the normality of errors. If the errors are distributed as per the Laplace distribution,...
ordinary least circles's user avatar
0 votes
0 answers
28 views

Simple OLS Regression Assumptions

Forgive me; I am new to learning statistics. I am trying to understand OLS simple linear regression better and gain an intuition for the associated assumptions. Upon my research, I have identified ...
LateGameLank's user avatar
3 votes
1 answer
248 views

A model suffering from omitted variable bias can be said to be unidentified?

If my regression model $$ y_i = \alpha + \beta x_i + \epsilon_i $$ suffers from OVB the error contains one variable which we assume correlated with $$ \epsilon_i = \gamma w_i + u_i $$ my estimate of $\...
Three Diag's user avatar
2 votes
3 answers
91 views

Deriving $k_i$ for $\hat{\beta} = \sum_{i=1}^n k_i y_i$ where $\hat{\beta}$ is the OLS -estimator

This question relates to this post: Prove that the OLS estimator of the intercept is BLUE Since I can't yet directly comment under the original post so I post my question here. In the original post ...
Roger Jia's user avatar
2 votes
1 answer
54 views

Question about OLS estimator (BLUE proof)

We know that the OLS estimator of $\beta$ is the unique BLUE. The proof goes as follows. Consider the general linear estimator $$\hat{\boldsymbol{\beta}}_\mathbf{A} = \hat{\boldsymbol{\beta}}_\text{...
framago's user avatar
  • 123
12 votes
3 answers
963 views

What does it mean for observations to be uncorrelated and have constant variance?

I am learning about linear regression from the textbook Elements of Statistical Learning by Friedman, Tibshirani, and Hastie. In this section they suppose we have a set of training data $(x_1, y_1), \...
CBBAM's user avatar
  • 311
0 votes
0 answers
49 views

Error in Bayesian Derivation of Covariance Matrix in Least Squares

I know variants of this question have been asked a million times, but rather than just asking "how do I derive the covariance matrix" I ask you to check the error in my calculations, because ...
geo's user avatar
  • 372
1 vote
0 answers
35 views

Small sample MLE vs OLS efficiency

MLE estimates are asymptotically efficient. Both MLE and OLS estimates are asymptotically normal and for many distributions their limiting variances coincide (information for one observation being the ...
memeplex's user avatar
  • 141
3 votes
1 answer
107 views

Kalman Filter to minimize weighted errors on the states: what's wrong with my derivation

I am thinking about how to implement a "weighted Kalman Filter". Note that the weights here are on the states. Basically the classical KF minimizes $\sum (x_i - \hat{x_i} )^2$ but I want to ...
Taylor Fang's user avatar
2 votes
0 answers
24 views

First and second moments of OLS slope when Conditional Expectation Function is not linear in covariates

Suppose I have a joint distribution of "outcomes" and "covariates", $(Y,X)$. Define the slope of the population "best" linear predictor of $Y$ given $X$ as $$\beta = \...
stats_model's user avatar
  • 2,485
0 votes
1 answer
26 views

Is it appropriate to use R² on filtering data?

At work, someone has built a dashboard to identify individuals likely to have higher value of the output variable. The approach involves fitting a OLS and measuring the R² value. They attempt to ...
B_fig's user avatar
  • 1
3 votes
2 answers
56 views

Perform a weighted linear regression on $x_i, y_i$ by doing a standard linear regression on $X_i, Y_i$?

Let's say we want to do a weighted linear regression between two series $(x_i)$ and $(y_i)$, with weights $(w_i)$, and get the coefficients from the line $y = mx + p$, and the $r^2$ coefficient. Is ...
Basj's user avatar
  • 632
1 vote
1 answer
76 views

What weighted average is OLS?

How can we find what sort of weighted-average a covariate is from a linear regression fitted with least squares? Consider the model $$ Y = \beta_0 + \beta_1 \tau + \sum_{j = 2}^k \beta_{j} X_{j} + \...
num_39's user avatar
  • 1,761
0 votes
0 answers
25 views

OLS Coeff importance of variables

I am using and OLS model to determine the importance of independents on the dependant. All variables are scaled. I am currently using the coeff as follows : Independent 1 coeff = 0.04 Independent 2 ...
milo204's user avatar
0 votes
0 answers
24 views

Granular regression with repeating dependent variable within group

I am estimating a standard OLS regression model where the unit of observation is inventor-firm-year level. The dependent variable I am interested in is patent count (a measure of inventor productivity)...
kurofune's user avatar
0 votes
0 answers
17 views

How can uncertainties in a generative model be propagated to an overall log-likelihood?

I am trying to use a Bayesian approach to carry out model selection and estimate the posterior distributions for parameters in a peak fitting scenario (quasi elastic neutron scattering). The ...
Andrew Nelson's user avatar
0 votes
0 answers
23 views

Graphical analysis of residuals

For an electricity experiment, I have collected data on the charging and discharging of a capacitor. Using the data and scipy.curve_fit, I have fitted the theoretical models. Now, I need to validate ...
Iván Solich's user avatar
0 votes
1 answer
48 views

statsmodels: Update OLS' degrees of freedom when absorbing 3+ fixed effects

I want to run an OLS regression with 3+ fixed effects. (Whether this is a good idea is out of the scope of this question). I can do it using Stata: ...
ebosi's user avatar
  • 138
0 votes
0 answers
19 views

OLS Model with Lags - logged coeff

i am building a OLS model using python, where the dependant and independent variables are lagged. This is a form of econometrics model where i want to figure out how much each independent variable ...
milo204's user avatar
1 vote
0 answers
18 views

How to estimate a feasible Generalized Error-in-Variable Model (combine deming regression/TLS and f-generalized least squares)

I have observational data with spatial structure. A hypothetical dataset could be brain mass for 100 species of birds and body mass for those same species. The data has spatial structure because ...
A Friendly Fish's user avatar
2 votes
1 answer
53 views

How Does Serial Correlation Cause OLS to remain unbiased (even in cross -sectional data)

In order for the coefficient estimators to remain unbiased in OLS, the conditional expectation of errors given the regressors needs to be zero, $E(u_i |x_i )=0$. However, if we have serial correlation ...
Jonathan Lee's user avatar
1 vote
0 answers
32 views

Would test set MSE and r^2 be the same with OLS and PC Regression with all PCs? [duplicate]

For this question, I define: PC Regression = Standardize Variables, Fit PCA, then apply OLS to all PCs. OLS = Standardize Variables, then apply OLS with all variables. This question makes me think the ...
user2330624's user avatar
0 votes
1 answer
62 views

Matrix decomposition with constraints and weighted least squares

We have a matrix, $\mathbf{X}$, of probability distributions between 6 different results, so each row $\mathbf{x}_i$ sums to 1. We want to perform dimension reduction so that each row is a linear ...
jgf1123's user avatar
1 vote
0 answers
45 views

Question on nonlinear least squares

Consider the following equation for $Y>0$: $$ (1) \quad \log(Y)=\log(\gamma)+\log(\alpha+\beta X)+\epsilon. $$ Assume that $E(\epsilon| X)=c\neq 0$. What are the consequences of this assumption on ...
Star's user avatar
  • 889
0 votes
0 answers
25 views

Is there a bias in linear least squares when all measurements have noise?

Imagine that we have a linear dependence $y = kx$. However our measurements $(\hat x_i, \hat y_i)$ have noise in both $x$ and $y$ $\hat x_i = x_i +\nu_i \ , \qquad \hat y_i = y_i +\varepsilon_i \ , \...
John Smith's user avatar
3 votes
2 answers
172 views

In linear regression, what's the asymptotic distribution of the error variance estimator?

Suppose $$Y_i=X_i'\beta+\epsilon_i$$ with $E(\epsilon_i|X_i)=0$ and $E\epsilon^2_i=\sigma^2$ and I estimate $\sigma^2$ using $s^2=\frac{1}{n}\sum_{i=1}^n (Y_i-X_i'\widehat{\beta})^2$, where $\widehat{...
ExcitedSnail's user avatar
  • 2,966
2 votes
0 answers
33 views

Regression with small sample size - LASSO or remove variables?

I'm trying to run a regression, but I only have 14 observations, each being a different city in the US. My dependent variable is the total number of trips per capita, and my explanatory variables are ...
BeyondConfused's user avatar
6 votes
1 answer
358 views

Is OLS asymptotically the best estimator even without gaussian error?

It is known that MLE is consistent and asymptotically efficient. OLS under certain assumptions is asymptotically normal. If the errors are gaussian, then OLS is equivalent to MLE. If the errors are ...
user405777's user avatar
1 vote
0 answers
13 views

Positive distance weighting

I have an overdetermined linear system of equations that's solved with least squares. I'd like to weight the equations to penalize a bunch of inputs clumped up together. Ideally if two (or more) ...
Marsupilami's user avatar
1 vote
0 answers
32 views

Fitting the rotation between two sets of 3D points, given 1D measurements

Context: I am measuring a series of points on the surface of an object, with a measuring device which can only capture the position of a point perpendicular to the the surface being measured. I am ...
rr-mark's user avatar
  • 11
0 votes
1 answer
38 views

Understanding differences in collinearity across Stata commands

Here's a simple example of a regression of y on x including time and id fixed-effects and both a linear and a quadratic time trend. If my t starts at 1, these 3 different regressions get the same ...
why's user avatar
  • 15
2 votes
1 answer
79 views

Granular difference-in-differences with non-repeating unit of observation

I want to analyze changes in characteristics of job postings around an (exogenous) event. However, rather than conducting the analysis at the job poster level (e.g., a company or geographic area), my ...
kurofune's user avatar
0 votes
0 answers
11 views

Does anyone have a name for a model that is by nature in a feedback loop?

I'm hoping someone could help me find some literature on a situation I'm dealing with, even if it's just by providing a name for what the system is called. Also, it is entirely possible I'm imagining ...
Nye307's user avatar
  • 11
2 votes
1 answer
52 views

Special case of Frisch-Waugh-Lowell theorem

The formulation is just a special case of FWL. Say we have a partitioned regression, $Y=X_1\beta_1+X_2\beta_2+\epsilon$ but with $X_2$ be $n\times 1$ and $\beta_2$ a constant. Let $b_1,b_2$ be two OLS ...
Chang Henry's user avatar
0 votes
0 answers
37 views

OLS - Visual Interpretation of var(β^​0​) [duplicate]

Hi guys! Could someone please help me with the interpretation of x_i^2? the drawing below is what I saw on the lecture but I'm having some trouble understanding what it means and why the mean is 0 in ...
Lex's user avatar
  • 1
0 votes
1 answer
68 views

What properties must be verified for a simple OLS regression model with time series?

Let's say I have 3 time series variables, $(X_t)$, $(Y_{t})$, $(Z_{t})$ and I estimate the following model with OLS estimator (I believe this form is called ARDL) : $$X_{t} \quad = \quad \alpha_1 X_{t-...
Johannes Konrad's user avatar
1 vote
1 answer
36 views

Why does transforming our independent variable improve forecasting?

When doing OLS regression, I've heard that transforming an explanatory (or dependent)* variables can (sometimes) improve the forecasting ability of the model. Theoretically, why does this work? I've ...
Michael Jones's user avatar
0 votes
0 answers
7 views

How to analyze the driver of a flipped sign regression result?

I have a panel dataset uniquely identified by $i$ and $t$. I am interested in the relationship between $x$ and $y$ and I want to an OLS regression with unit fixed effects ($\phi_i$) as follows: $$y_{...
SXS's user avatar
  • 131
0 votes
0 answers
35 views

OLS - How to create a good feature

I'm working on trying to predict the next price return for a stock given the inside bid-ask volumes (level 1 data) and I want to create a feature that models the imbalance between bid volumes and ask ...
Joe's user avatar
  • 271
7 votes
3 answers
339 views

Different estimates of Least Squares and Maximum Likelihood Estimates under non-normality

It is said that Least Squares estimates would differ from Maximum Likelihood estimates if the underlying data were non-normal. This should be the reason why LS estimates can be used in linear ...
Anti's user avatar
  • 159
2 votes
1 answer
56 views

Why does `systemfit` yield different results for OLS and WLS under cross-equation restrictions?

I am following up on the question "Why does systemfit yield identical results for OLS and WLS?". It deals with estimating a system of linear equations ...
Richard Hardy's user avatar
2 votes
2 answers
79 views

Why does `systemfit` yield identical results for OLS and WLS?

I am estimating a system of seemingly unrelated regressions (SUR) using the systemfit package in R. Each of the equations has one unique regressor and one common ...
Richard Hardy's user avatar
2 votes
1 answer
39 views

Does OLS give the maximum likelihood estimation for a linear log model?

I'm fitting a model $y=a\times \log(x)+ b$ using standard scikit linear regression (wich uses OLS) and a transformation $x'=\log(x)$. My doubt is: the parameters I get for the model are the best one ...
Roger Danilo Figlie's user avatar
2 votes
2 answers
59 views

Test and Train RSS in OLS model

I encountered the following true/false question: Given a train sample with $\ N $ observations and OLS model fitted on that sample, the RSS of the train sample will be less than or equal to the ...
bm1125's user avatar
  • 187
1 vote
1 answer
33 views

How can I find what is driving the change between my updated model and the original model?

I'm working on updating a model of rents (which is currently simple OLS) for my employer who has a large national portfolio. By tweaking here and there and drawing in a large amount of exogenous data ...
wjb_hwe's user avatar
  • 11
4 votes
3 answers
125 views

Calculate $R^2$ given estimated coefficients and $N$ only

We have a simple regression equation $y=a+bx$, where $a,b$ were estimated via OLS -- we know these values. Suppose the number of observations $N=25$ is given. Is it true, that we cannot calculate $R^2$...
Vnature's user avatar
  • 345
0 votes
1 answer
48 views

Outliers in OLS

I'm running OLS regression in stata with 5-6 IVs (all dummy variables) and a DV (continuous). Does it make sense to check for outliers (e.g., I can't run scatter dv x1 x2) in this case? Also, is it ...
brian's user avatar
  • 75
2 votes
1 answer
53 views

Regression in sex differences multiple dependent variables

If I want to examine sex differences in three variables, lets say academic attainment, study motivation, and a variable that is categorical. How should can I fit these variables with OLS regression? ...
user409571's user avatar

1
2 3 4 5
58