Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

0
votes
0answers
12 views

Ridge least squares in python. statsmodels vs sklearn vs closed form

I'm having a bit of an issue understanding on how to use Ridge regularization with statsmodels. I tried running the same model using 3 different methods and while I get consistent results between the ...
-4
votes
1answer
57 views
3
votes
0answers
28 views

Dealing with autocorrelation using Generalized Least Squares

I have a time series data set where the auto correlation of the residuals follow an exponential decay. I was wondering how I should deal with this? I would like to fit a linear model and have tried ...
0
votes
1answer
19 views

Is it possible to add the standard errors of 2 groups together to obtain the standard error of the 2 groups combined

I am trying to recreate the results in this table. The results have been obtained by difference in difference estimation. I can obtain values from all columns except for column 5 and 6. Column 5 says ...
1
vote
0answers
30 views

OLS Population Orthogonality Condition Proof

In the OLS model, we assume that $E(X'U)=0$ (with $u$ being the error term), which comes from $E(U|X=x)=0$, providing us that $E(U)=0$ and $cov(x_i, u)=0$ $\forall x_i$. I understand this argument ...
0
votes
0answers
24 views

Ordinary least squares does not optimize error

I'm trying to use polynomial regression to fit the curve $$X = [0, 1]$$ $$Y = \sin(2 \pi X) + \epsilon$$ where $\epsilon$ is normally distributed with the same $\sigma$ for all $X$ For every value ...
2
votes
0answers
21 views

Baseline adjustment in a change model given bidirectional causation

I come from a non-statistical background and am trying to wrap my head around whether baseline adjustment is necessary in a change model when analysed using OLS regression. I am considering different ...
1
vote
1answer
28 views

Uncorrelated errors with the regressor in a reduced form VAR

I have a reduced form VAR $$\begin{equation} y_t = c_o + A y_{t-1} + \epsilon_t \end{equation}$$ Where, $y_t \in \mathbb{R}^2$, $A$ is a $2$X$2$ matrix and $$\begin{equation} E(\epsilon_t \...
2
votes
1answer
65 views

Controlling for a variable in OLS - Stratification and Reaggregation. Simple Example

In his engrossing book "Naked Statistics" Charles Wheelan begins to explain how controlling for variables works by stratifying the sample. However, he stops short of explaining the reaggregation, ...
0
votes
0answers
23 views

Solving correlation between explanatory variables using instrumental variables

I am currently stuck on a task where I am interested in estimating the production function for agricultural output as follows: \begin{equation} y_{i} = x_{i}\beta + \alpha_i + \epsilon_{i} \end{...
0
votes
0answers
12 views

Transforming panel data OLS into cross-sectional data model

I am currently stuck on a task where I am interested in estimating the production function for agricultural output using panel data as follows: \begin{equation} y_{it} = x_{it}\beta + \alpha_i + \...
0
votes
0answers
17 views

R square for fixed model is worse than OLS in panel model

I am analysing panel data and using the plm package in R. I'm using the plmtest, pFtest, phtest functions which indicate to me that fixed effects model should be used over the pooled OLS and random ...
1
vote
0answers
18 views

Feature Selection with Branch & Bound in Python [closed]

Which useful python packages should be used to perform feature selection using Branch & Bound? The Branch & Bound algorithm has been proposed by Furnival and Wilson (1974) and does not look at ...
1
vote
0answers
15 views

Proof of variance of ridge estimate with only one predictor!

Let's consider Ridge with only one predictor (extreme and simple case). I would like to proof that $V(B_r)=\sigma^2/(1+\lambda)$, so its variance it less than OLS variance, that is $V(B_{OLS})=\sigma^...
1
vote
0answers
39 views

Time Series OLS

There are 2 time series $X$ and $Y$ and 3 sets: the first set consists of $N_1$ observations, the second set contains $N_2$ observations right after the first set, and third set contains $N_1$ and $...
0
votes
0answers
22 views

Pooled OLS to achieve a consistent estimate [duplicate]

The task is to estimate the production function for agricultural output. Theoretically I have access to panel data. The production function I want to estimate is: \begin{equation} y_{it} = x_{it}\...
2
votes
1answer
64 views

Selecting between OLS regression and ARIMA for time series, why AIC or BIC for ARIMA is much larger in my data?

My data set is quarterly time seires data (around 140 data points). Method 1: simple OLS regression with 5-6 exogenous variables, which are drivers of the dependent variable. None of the explanatory ...
0
votes
0answers
18 views

Random effects model vs pooled OLS

Can someone please explain why bother using random effects if the unobserved constant effects are assumed to not be correlated with the explanatory variable? Why not just using a pooled OLS?
0
votes
1answer
36 views

How to check for independence of errors?

How to check for independence of errors in OLS regression? Let's say I have 10 observations for each hour. If I plot residuals ordered by time, I have the problem that adjacent residuals refer to the ...
1
vote
0answers
10 views

Autocorrelation with Replicates

How to perform autocorrelation with replicates? For each day I have many observations and I want to check wether these observations are correleated with the next day and the day after this day. If if ...
2
votes
2answers
274 views

Why use OLS when it is assumed there is heteroscedasticity?

So I'm slowly going through the Stock and Watson book and I'm a bit confused on how to deal with the issue of homoscedacity/heteroscedacity. Specifically, it is mentioned that economic theory tells ...
0
votes
0answers
10 views

Independence of Residuals: Multiple Measures for each Point in Time

One assumption of OLS regression is independence of residuals. I'm not sure how this assumption can be checked for the following study design. Every day 5 measurements were carried out, for which I ...
0
votes
0answers
23 views

VAR/VEC model selection

I want to model a relationship between a good's price and a few variables using time-series data. I run VEC/VAR models and get a series of equations. My question is how to use these results (using ...
0
votes
0answers
22 views

Mediation effect - violated assumptions regarding normally distributed errors and homoscedasticity

I am doing mediation analysis using macro developed by Hayes. Since the macro uses bootstrapping for the estimation of coefficients it still is based on OLS. I have not find clear answer regarding the ...
0
votes
0answers
16 views

Estimating Fourier parameters using least squar emethod

I'm given that $$ \sum_{t=1}^N \epsilon^2(t) = \sum_{t=1}^N\left[y(t) - \sum_{n=0}^{N/2}\left\{a_n\cos\left(\frac{2\pi nt}{N}\right) + b_n\sin\left(\frac{2\pi nt}{N}\right)\right\}\right]^2 $$ and ...
0
votes
1answer
28 views

Bias of omitting squares and interactions

With $x_1\sim N(\mu,\sigma^2)$ and a population model... $Y=\alpha_0+\alpha_1X_1+\alpha_2X_1^2+\epsilon$ ...if I run OLS omitting the square term... $y_i=\beta_0+\beta_1x_{1,i}+u_i$ ...the $x_1$ ...
0
votes
0answers
17 views

Calculate 95% confidence interval for profile likelihood

I'm not at all a statistician, so please bear with me. I have a mathematical system $x' = f(x,P)$, where $P$ is the set of parameters that I try to estimate and $x = (x_1,x_2,x_3,x_4,x_5) \in \mathbf{...
0
votes
1answer
30 views

Minimize Logged Sum of Squares?

When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
0
votes
0answers
27 views

Connections between pseudoinverse, linear regression, BLUE, ordinary least square

They all take similar forms, why is that, what are the connections here? pseudoinverse: $Ax=b, x=(A^TA)^{-1}A^Tb$ linear regression: $ \hat{y}=x^T(X^TV^{-1}X)^{-1}X^TV^{-1}y$, where X is the data, y ...
0
votes
0answers
25 views

Before-During-After Regression Analysis

I am currently trying to study the effect of the prolonged absence of the MVP in a basketball team on the performance of the team as a whole. Specifically, the idea was to create a regression model ...
1
vote
1answer
37 views

Endogeneity - Omitted variable bias in OLS

If a have a true model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$ but $x_3$ is unobservable. What are the consequences of having a unobservable variable which correlates ...
0
votes
0answers
23 views

Performing a cross-sectional regression

This might seem a very fundamental question, but I am having issues grasping how a cross-sectional regression is performed? I have approx. 40 million data points over a period of 21 years, wherein I'd ...
1
vote
0answers
35 views

Linear least squares algorithms

I have stumbled across these two questions and accepted answers: (1) Do we need gradient descent to find the coefficients of a linear regression model? (2) Why use gradient descent for linear ...
1
vote
1answer
36 views

Diff-in-Diff special type of OLS?

Is difference-in-differences just a special type of OLS? Can I add fixed effects in my diff-in-diff model?
1
vote
0answers
17 views

Lagged Dependent variable in OLS

I have a question about one of my models. I am sorry if I am using Terms wrongly, as I am part of the management research field and this quite often leads to different terminologies. I try to model ...
3
votes
1answer
75 views

Errors and residuals in linear regression

I think in common literature about statstics the authors are often very imprecise when it comes to residuals and errors. So far, I could not work that difference out completely and therefore have ...
1
vote
0answers
18 views

How to compute the “hat-matrix” of constrained least squares

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $...
1
vote
0answers
15 views

Partially-Linear Least Squares

Model I'm working with univariate time series data $(y_1, \dots, y_n)$ where time $t \in [1, n]$. Suppose the mean function has a known form, in my case $$\mathrm{E}(Y_t) = 1 - \alpha e^{-t/\tau_1} - ...
2
votes
2answers
62 views

When do you use logistic regression vs. when you do use OLS?

If you are creating a regression model where the response variable is a numerical value, but one of the variables is a dummy (binary), can you use OLS-method? Do you only use logistic regression if ...
0
votes
0answers
21 views

Choice of residual function for least squares error minimization

Good morning, I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data. I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as $K_{IC} = \sigma \sqrt{D} k_0(\...
2
votes
3answers
221 views

OLS effect of X squared ond dependent variable

I have my OLS regression: $y = \beta_1 + \beta_2 X_2 + \beta_3 X_3 +\beta_4 (X_3)^2$ Could anybody please explain to me the effect of a change in $X_3$ on the dependent variable?(Is the effect ...
0
votes
0answers
38 views

Can we unify generalized linear models and ordinary least squares by switching between two metric spaces

Lots of smart people out there. Maybe someone has seen this concept. In linear regression using ordinary least squares (OLS) we simply project the response Y onto the range of the design matrix X. ...
0
votes
0answers
7 views

how to systematically choose the thresholding in least square solver, for solving ill-posed least square problem?

I have a system $Ax =b$, where $A\in\mathbb{R}^{300\times 200}$, but $rank(A)=70$. And I know the true solution. I tried standard least square solver such as ...
2
votes
1answer
27 views

Is constrained (nonnegative) least squares a form a regularisation?

Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
6
votes
1answer
67 views

What is the difference between $E[\varepsilon\mid X]=0$ and $E[\varepsilon X]=0$ in OLS regression?

Why is the assumption $E[\varepsilon X]=0$ weaker than $E[\varepsilon\mid X]=0$?
0
votes
1answer
41 views

Showing that the minimum-variance estimator is the OLS estimator

Recap of required theory Consider the following regression: $$y_i = \alpha + \beta x_i + u_i \tag{1}$$ where $y_i$ are iid and $x_i$ are deterministic (i.e. fixed). We know that the OLS estimator $...
0
votes
1answer
32 views

How to use White-test to check if the heteroscedasticity has been effectively dealt with by a WLS?

Consider an OLS model with $n$ observations and $p$ explanatory variables (including an intercept term) $$y=X\beta + \epsilon$$ We may use a White test to (approximately) check for the presence of ...
2
votes
1answer
63 views

Transform data while keeping the mean constant

I am using an OLS regression to fit a model to some data. The estimated response is given by the usual $$\mathbf{\hat{y}} = \mathbf{Py} = \mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}...
4
votes
1answer
179 views

Sufficient Statistic for $\beta$ in OLS

I have the classical regression model $$y = \beta X + \epsilon$$ $$\epsilon \sim N(0, \sigma^2)$$ where $X$ is taken to be fixed (not random), and $\hat\beta$ is the OLS estimate for $\beta$. It is ...
0
votes
0answers
32 views

Including both state, time and state-time fixed effects in a regression

My question relates to the use of interacted fixed effects (say suburb-time) together with non-interacted fixed effects (suburb, and then time) in the same regression. Specifically, imagine we had a ...