Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

0
votes
0answers
9 views

LS classifier that is based on the sum of error squares criterion

Is it possible given a set of points from two classes to determine the LS classifier that is based on the sum of error squares criterion? I don't ask if it is efficient if it is possible and if yes ...
0
votes
0answers
106 views

How to prove variance of OLS estimator in matrix form?

I am reading Wooldridge's Introductory Econometrics (2000), don't judge me, old version = cheap second hand book, and in the page P94 Theorem 3.2 of Multiple Regression Analysis, it says that: $$ Var(...
2
votes
0answers
70 views

Negative values for OLS variance

I am currently writing some code which performs regression and have noticed that when I calculate variance of $c\hat{\beta}$ I am sometimes on some datasets getting negative values. The variance is ...
0
votes
1answer
82 views

Implications of strict exogeneity for OLS in time series

Zero Conditional Mean (ZCM), or Strict Exogeneity, is given by: $E[u|X]=0$ Equivalently, $E[u_t|X]=0, t=1,...,T$ Is it true that this implies: Zero Unconditional Mean: $E[u_t]=0, \forall t$ ...
2
votes
0answers
36 views

Comparison between coefficients of two fixed effects models with same sample

I have a problem. I have two models of OLS regressions with time and group fixed effects of a crosscountry paneldata analysis: $$Y_i = b_1 X_i + b_2 D_1 Z_i + b_3 D_2 Z_i\tag 1$$ and $$Y_i = b_1 ...
0
votes
0answers
34 views

Does standard error affected by the coefficient?

I make a comparison on ridge regression and OLS using simulation. As i set my correlation as 0.9, which is high, i expect the standard error of ridge regression to be low. However, it is not. ...
2
votes
0answers
32 views

Intuition behind White's Estimators/ Heteroscedasticity-consistent Standard Errors

For a medical study I am trying to understand the intuition behind heteroscedasticity-consistent standard errors. I know that it can be used, when in OLS regression residuals are heteroscedastic. By ...
3
votes
1answer
37 views

Obtain global linear regression estimate from subsamples

I want to estimate $\widehat\beta$ in a simple linear regression with scikit. $$y = X \beta + \varepsilon$$ The problem is that the dimension of the complete $X$ is too large to fit into memory. Is ...
0
votes
0answers
19 views

Comparing the mean of predicted values for a misspecified model against the mean of the observed values

I have a regression that I have run on average ratings for some products (dependent variable) and their characteristics (Model 1). I have reason to believe there is a prejudice against a specific set ...
1
vote
0answers
91 views

How to verify the “random sampling” Gauss-Markov Assumption with Stata (or anything else)?

According to the book I am using, Introductory Econometrics by J.M. Wooldridge, there are 5 Gauss-Markov assumptions necessary to obtain BLUE. However, by looking in other literature, there is one ...
1
vote
1answer
58 views

linear regression predicts lower than expected

I am trying to predict first term GPA for college students based on a number of incoming factors (high school gpa, placement test, year). This isn't the overall model just a simpler one. The first ...
2
votes
2answers
38 views

Linearization of multiplicative and exponential regression models and their OLS estimation

So far I have been under the impression that you can "linearize" multiplicative models of the form (1) $y=\alpha * \beta_1x_1 * \beta_2x_2 * \beta_3x_3 $ and exponential models of the form (2) $y=\...
1
vote
0answers
15 views

Simulating size distrotion with a VAR(1) model

Firstly, I am not asking for code, I would like the intution of how I would do this. I am testing for size distortion. I have estimated and VAR(1) model and I have the parameters. I want to ...
1
vote
0answers
32 views

Consistency of an OLS estimator in time series

In the ARMA(2,1) time series $y_t = \beta_0+\beta_1y_{t-1}+\beta_2y_{t-2}+ u_t + \phi_1u_{t-2}$ $u_t$ and $u_{t-2}$ are white noise shocks. The time series are stationary and ergodic. In the ...
2
votes
0answers
26 views

Residualizing in OLS

Consider a linear regression $$ y_i = A_i' \theta + B_i' \psi + \epsilon_i $$ There's a "trick" to find the parameters $\theta$ and $\psi$ in a stepwise fashion. First minimizing squared error wrt....
0
votes
1answer
34 views

Without testing: when are error terms possibly homoscedastic?

I am facing the following study: In the 1980's, Tennessee conducted an experiment in which kindergarten students were randomly assigned to regular and ...
0
votes
1answer
29 views

How to fix and understand linearity

The model I have run is a simple multiple linear regression. The model looks like a great fit, but R is telling me otherwise. My question is 3 fold. 1) How do we estimate linearity (not visually) 2) ...
2
votes
0answers
42 views

Can I average out a constant (intercept) in OLS regression?

I have a OLS regression in the form: $$Y_t=\alpha +\beta X_{t-1}+\varepsilon_{t}$$ Can I average out the constant during the OLS estimation/derivation and report, $$y_t=\beta X_{t-1}+\varepsilon_{...
3
votes
0answers
58 views

Dealing with autocorrelation using Generalized Least Squares

I have a time series data set where the auto correlation of the residuals follow an exponential decay. I was wondering how I should deal with this? I would like to fit a linear model and have tried ...
0
votes
1answer
25 views

Is it possible to add the standard errors of 2 groups together to obtain the standard error of the 2 groups combined

I am trying to recreate the results in this table. The results have been obtained by difference in difference estimation. I can obtain values from all columns except for column 5 and 6. Column 5 says ...
1
vote
1answer
82 views

OLS Population Orthogonality Condition Proof

In the OLS model, we assume that $E(X'U)=0$ (with $u$ being the error term), which comes from $E(U|X=x)=0$, providing us that $E(U)=0$ and $cov(x_i, u)=0$ $\forall x_i$. I understand this argument ...
0
votes
0answers
26 views

Ordinary least squares does not optimize error

I'm trying to use polynomial regression to fit the curve $$X = [0, 1]$$ $$Y = \sin(2 \pi X) + \epsilon$$ where $\epsilon$ is normally distributed with the same $\sigma$ for all $X$ For every value ...
2
votes
0answers
43 views

Baseline adjustment in a change model given bidirectional causation

I come from a non-statistical background and am trying to wrap my head around whether baseline adjustment is necessary in a change model when analysed using OLS regression. I am considering different ...
1
vote
1answer
41 views

Uncorrelated errors with the regressor in a reduced form VAR

I have a reduced form VAR $$\begin{equation} y_t = c_o + A y_{t-1} + \epsilon_t \end{equation}$$ Where, $y_t \in \mathbb{R}^2$, $A$ is a $2$X$2$ matrix and $$\begin{equation} E(\epsilon_t \...
3
votes
1answer
106 views

Controlling for a variable in OLS - Stratification and Reaggregation. Simple Example

In his engrossing book "Naked Statistics" Charles Wheelan begins to explain how controlling for variables works by stratifying the sample. However, he stops short of explaining the reaggregation, ...
0
votes
0answers
31 views

Solving correlation between explanatory variables using instrumental variables

I am currently stuck on a task where I am interested in estimating the production function for agricultural output as follows: \begin{equation} y_{i} = x_{i}\beta + \alpha_i + \epsilon_{i} \end{...
0
votes
0answers
21 views

Transforming panel data OLS into cross-sectional data model

I am currently stuck on a task where I am interested in estimating the production function for agricultural output using panel data as follows: \begin{equation} y_{it} = x_{it}\beta + \alpha_i + \...
0
votes
0answers
32 views

R square for fixed model is worse than OLS in panel model

I am analysing panel data and using the plm package in R. I'm using the plmtest, pFtest, phtest functions which indicate to me that fixed effects model should be used over the pooled OLS and random ...
1
vote
0answers
24 views

Feature Selection with Branch & Bound in Python [closed]

Which useful python packages should be used to perform feature selection using Branch & Bound? The Branch & Bound algorithm has been proposed by Furnival and Wilson (1974) and does not look at ...
1
vote
0answers
39 views

Proof of variance of ridge estimate with only one predictor!

Let's consider Ridge with only one predictor (extreme and simple case). I would like to proof that $V(B_r)=\sigma^2/(1+\lambda)$, so its variance it less than OLS variance, that is $V(B_{OLS})=\sigma^...
1
vote
0answers
47 views

Time Series OLS

There are 2 time series $X$ and $Y$ and 3 sets: the first set consists of $N_1$ observations, the second set contains $N_2$ observations right after the first set, and third set contains $N_1$ and $...
0
votes
0answers
29 views

Pooled OLS to achieve a consistent estimate [duplicate]

The task is to estimate the production function for agricultural output. Theoretically I have access to panel data. The production function I want to estimate is: \begin{equation} y_{it} = x_{it}\...
2
votes
1answer
72 views

Selecting between OLS regression and ARIMA for time series, why AIC or BIC for ARIMA is much larger in my data?

My data set is quarterly time seires data (around 140 data points). Method 1: simple OLS regression with 5-6 exogenous variables, which are drivers of the dependent variable. None of the explanatory ...
0
votes
0answers
77 views

Random effects model vs pooled OLS

Can someone please explain why bother using random effects if the unobserved constant effects are assumed to not be correlated with the explanatory variable? Why not just using a pooled OLS?
0
votes
1answer
52 views

How to check for independence of errors?

How to check for independence of errors in OLS regression? Let's say I have 10 observations for each hour. If I plot residuals ordered by time, I have the problem that adjacent residuals refer to the ...
1
vote
0answers
12 views

Autocorrelation with Replicates

How to perform autocorrelation with replicates? For each day I have many observations and I want to check wether these observations are correleated with the next day and the day after this day. If if ...
2
votes
2answers
304 views

Why use OLS when it is assumed there is heteroscedasticity?

So I'm slowly going through the Stock and Watson book and I'm a bit confused on how to deal with the issue of homoscedacity/heteroscedacity. Specifically, it is mentioned that economic theory tells ...
1
vote
1answer
36 views

Bias of omitting squares and interactions

With $x_1\sim N(\mu,\sigma^2)$ and a population model... $Y=\alpha_0+\alpha_1X_1+\alpha_2X_1^2+\epsilon$ ...if I run OLS omitting the square term... $y_i=\beta_0+\beta_1x_{1,i}+u_i$ ...the $x_1$ ...
0
votes
0answers
24 views

Calculate 95% confidence interval for profile likelihood

I'm not at all a statistician, so please bear with me. I have a mathematical system $x' = f(x,P)$, where $P$ is the set of parameters that I try to estimate and $x = (x_1,x_2,x_3,x_4,x_5) \in \mathbf{...
0
votes
1answer
38 views

Minimize Logged Sum of Squares?

When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
1
vote
1answer
66 views

Endogeneity - Omitted variable bias in OLS

If a have a true model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$ but $x_3$ is unobservable. What are the consequences of having a unobservable variable which correlates ...
1
vote
0answers
38 views

Linear least squares algorithms

I have stumbled across these two questions and accepted answers: (1) Do we need gradient descent to find the coefficients of a linear regression model? (2) Why use gradient descent for linear ...
1
vote
1answer
111 views

Diff-in-Diff special type of OLS?

Is difference-in-differences just a special type of OLS? Can I add fixed effects in my diff-in-diff model?
1
vote
0answers
109 views

Lagged Dependent variable in OLS

I have a question about one of my models. I am sorry if I am using Terms wrongly, as I am part of the management research field and this quite often leads to different terminologies. I try to model ...
3
votes
1answer
101 views

Errors and residuals in linear regression

I think in common literature about statstics the authors are often very imprecise when it comes to residuals and errors. So far, I could not work that difference out completely and therefore have ...
1
vote
0answers
31 views

How to compute the “hat-matrix” of constrained least squares

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $...
1
vote
0answers
15 views

Partially-Linear Least Squares

Model I'm working with univariate time series data $(y_1, \dots, y_n)$ where time $t \in [1, n]$. Suppose the mean function has a known form, in my case $$\mathrm{E}(Y_t) = 1 - \alpha e^{-t/\tau_1} - ...
2
votes
2answers
211 views

When do you use logistic regression vs. when you do use OLS?

If you are creating a regression model where the response variable is a numerical value, but one of the variables is a dummy (binary), can you use OLS-method? Do you only use logistic regression if ...
2
votes
3answers
234 views

OLS effect of X squared ond dependent variable

I have my OLS regression: $y = \beta_1 + \beta_2 X_2 + \beta_3 X_3 +\beta_4 (X_3)^2$ Could anybody please explain to me the effect of a change in $X_3$ on the dependent variable?(Is the effect ...
0
votes
0answers
7 views

how to systematically choose the thresholding in least square solver, for solving ill-posed least square problem?

I have a system $Ax =b$, where $A\in\mathbb{R}^{300\times 200}$, but $rank(A)=70$. And I know the true solution. I tried standard least square solver such as ...