Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

-1
votes
0answers
12 views
0
votes
0answers
17 views

Error in drawing normal distribution from Stata [on hold]

I have to draw a normal distribution for an OLS estimator I am using the command drawnorm betaest, n(10000) means(beta_mean) sds(betasq) However, it is giving me the error no; data in memory ...
2
votes
1answer
2k views

Prove that OLS estimator of the intercept has minimum variance

Let $$y_i=B_0+B_1X_i+\varepsilon_i$$ where $\varepsilon_i\sim N(0,\sigma^2)$. Find the least squares estimator of $B_0$ and show that it is unbiased and has minimum variance. I will not write in ...
1
vote
1answer
13 views

Upwardly biased forecast results due to period of high demand: how to deal with this?

I'm currently working on a call-center forecasting project with some data limitations. Currently it is still a learning-project, and I started with simple OLS regressions. For the months 2016-12 to ...
0
votes
0answers
24 views

Consistency of variance estimator in OLS [duplicate]

Given the model, $$ y_i = x_i'\beta + \epsilon_i \quad \epsilon_i \sim N(0, \sigma^2) \quad iid \quad \forall i = 1, ..,n $$ how can I prove that the estimator of the variance $\hat{\sigma}^2 = \...
1
vote
0answers
16 views

How I can sketch the proof of consistency of only one beta in multiple regression?

Now assume you additionally obtained data on average parental incomes (PI) and the ethnic composition (EC) of the pupils in school. You regress the score on STR PI EC and a constant. State the ...
2
votes
1answer
24 views

Conditions in which the sign of a coefficient will change between a linear probability model and a logistic model

I am estimating a model where the DV is a binary variable and the key independent variable is the interaction between a dummy variable and a continuous variable. I am getting an very odd result where ...
0
votes
0answers
18 views

How to prove that conditional distribution for Y given OLS for simple linear regression do not depends on original parameters?

How to prove that for a simple linear regression model: $$y_i=\beta_0+\beta_1 x_i+\varepsilon_i,$$ the conditional distribution $$Y|\hat{\beta}_0,\hat{\beta}_1$$ do not depends on $\beta_0$ and $\...
1
vote
0answers
17 views

how to prove B1 as a consistent estimator in panel data

$Yi=a+ B_1*X_i+ B_2*Z_i+\epsilon_i$, and suppose that $Zi$ is unobservable and not correlated with $X_i$. Is the OLS estimator of $B_1$ consistent by regressing $Y_i$ on a constant a and $X_i$? I ...
1
vote
0answers
8 views

How can I implement this Robust PCA equation in a more efficient way?

I recently learned in class the Principle Component Analysis method aims to approximate a matrix X to a multiplication of two matrices Z*W. If X is a n x d matrix, Z is a n x k matrix and W is a k x d ...
4
votes
1answer
1k views

When are the asymptotic variance of OLS and 2SLS equal?

Assume the model $ \ y = X\beta + u \ $ with $\ W \ $ is a $ \ n\times l \ $ so called matrix of instruments. The following assumptions hold. There is a law of large numbers (LLN) for 1.,2.,3. and 4....
0
votes
1answer
91 views

Estimating the variance of the GLS estimator

Consider the linear regression model where $y = XB + u$. Assume that $\mathrm{E}[u \mid X] = 0$. Assume that $\mathrm{V}[u \mid X] = \sigma^2I$. This is the simple linear model. Now relax the ...
0
votes
0answers
9 views

Regression using previous years results

Is it possible to make a linear model non auto regressive using previous years observation's. For example I have a student score, and I have 3 contributing variables (subject scores e.g. science......
1
vote
0answers
22 views

Second order gradient-based method to locally maximize sum of squares

Non-linear least squares algorithms such as Gauss-Newton allow me to (locally) minimize a sum of squares of residuals (the output of some non-linear function). I.e. locally solve: $$ \mathbf{x} = \arg\...
4
votes
2answers
343 views

What are the consequences of “copying” a data set for OLS?

Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where $$\hat{\...
1
vote
1answer
22 views

Loss function for KNN Regressor

What is the Loss function for KNN Regressor? Would it be similar to OLS? If so what would be the main difference?
0
votes
0answers
11 views

OLS Multicolleniarity

I have a pretty simple task to estimate ols multiple regression. I need a measure of multicolleniarity. Is condition number a good measure and what criteria exists fot its value?
1
vote
3answers
36 views

Question Regarding Zero Conditional Mean

Hi I am a beginner to econometrics! I have been dealing with bivariate regression. We use the formula $y = \beta_0 + \beta_1 x$. I am told that if $E(u\mid x) \ne 0$ then the estimate of the slope ...
0
votes
0answers
20 views

Difference among MLE, GMM and OLS [on hold]

I want to know what exactly the differences among OLS, GMM and MLE estimators and why statisticians prefer more to implement GMM while econometricians are more likely to use OLS or GMM. Since it is ...
0
votes
0answers
59 views

auto-correlation and OLS regression

I was trying to find the OLS estimator for the model: $Y$ = $\beta_0$ + $\beta_1X_{1t}$ + $\beta_2X_{2t}$ +.......+ $\beta_5X_{5t}$ + $e$ t = 1,2,3 ......, 50 time ordered observations X is a full ...
1
vote
1answer
36 views

A very basic question on ENDOGENEITY

In the regression model $Y$ = $\beta_0$ + $\beta_1X_1$ + $\beta_2X_2$ +.......+ $\beta_kX_k$ + $\epsilon$ where $\epsilon$ = $\delta_0X_2$ + $\lambda$ Will this also be the case of endogeneity ...
1
vote
1answer
193 views

How to find the best piecewise-smooth fit

I have data points $(\theta,y)$ that when plotted in a x-y (with $\theta$ being the x) graph can be fit by the curve $$y=-\alpha-\frac{\beta}{\cos(\theta_{0}-\theta)}$$ i.e. I can find $\alpha, \...
0
votes
0answers
14 views

Generalized Least Squares and Heteroskedasticity

I am trying to model a OLS where I know that the hetero-skedasticity is like this $E(\epsilon^2)$ = $\sigma_i^2$ = $\delta_0$ + $\delta_1*X_{i2}$ So, I was using the concept of feasible generalized ...
1
vote
1answer
33 views

Implications of i.i.d. sample

I have the following question: I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as $\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
0answers
6 views

Least square estimate for post-stratification sampling

I figured out via the normal linear regression method that Beta0 hat = ybar - Beta1 hat xbar. But I am not sure how to find out the least square estimate for Chat. Is anyone able to help me? Thanks!! ...
15
votes
1answer
2k views

Other unbiased estimators than the BLUE (OLS solution) for linear models

For a linear model the OLS solution provides the best linear unbiased estimator for the parameters. Of course we can trade in a bias for lower variance, e.g. ridge regression. But my question is ...
0
votes
1answer
23 views

Time as an independent variable in multiple regression

Can we chose time (t) as one of the independent variables in multiple (OLS) regression?
0
votes
1answer
216 views

impact of exchange rate and interest rate on stock market

I've conducted ols regression analysis on stock market index KLCI (dependent) and 2 other independent variables that includes exchange rate (ER) and interest rate (IR). And the result shows that i ...
1
vote
1answer
25 views

2-step linear regression

Very simple problem: First model: I run a linear regression of $Y$ on $X$ and $Z$. Second model: I regress $Y$ on $X$ only, compute the residuals, and regress these residuals on $Z$. Why do I ...
1
vote
2answers
235 views

Full-Rank design matrix from overdetermined linear model

I'm trying to create a full-rank design matrix X for a randomized block design model starting from something like the example from page 3/8 of this paper . It's been suggested that I can go about ...
0
votes
1answer
29 views

Least squares regression coefficient with minimal information

If I only have a correlation matrix of 4 variables and the sample size, is it possible to predict 1 variables from the other 3 while using information about sample size? I’m trying to use lm but my ...
2
votes
1answer
132 views

Learning hat matrix

I'm stuck learning the hat matrix and wondered if someone could help with a question. If I have the model $$Y_i =\beta_0+\beta_1X_i+\epsilon_i,i = 1,2,3 \dots n,$$ how can I calculate the hat matrix ...
1
vote
1answer
54 views

OLS estimation of intercept in AR($p$) in R

I investigate the performance of the OLS estimator of an AR($3$) model given by $$ X_t=\mu+\phi_1X_{t-1}+\phi_2X_{t-2}+\phi_3X_{t-3}+\varepsilon_t $$ for $t\in\mathbb Z$ using the following code: <...
0
votes
0answers
19 views

relation between OLS regressions using different data transformations

I have a $(n \times d)$ panel $y$ of $n$ different variables , and a $(n \times d)$ panel $x$ of their forecasts. $d=$ time length of data $n=$ cross section width/ no. of variables I run a pooled (...
0
votes
0answers
9 views

Autocorrelated independent variable (how to treat)

I have some response variable $Y$ that I am trying to model with OLS. There is a single independent variable $X_{t,N}=Y_t-Y_{t-N}$ for some $N$. However, when I do such a simple regression, I obtain ...
0
votes
0answers
21 views

OLS Characteristics - Proof

I am trying to figure out the mathematical proof of one of the OLS characteristics in simple linear regression. One stage, or more accurately, one term is not working out. The term coloured in red is ...
2
votes
1answer
742 views

AR.OLS isn't matching to an OLS on the autoregressive lags, Why?

I am using R and running ar.ols() on some data. And trying to compare to a more "manual" method of computing an AR model by doing lm() using the autoregressive lags as my independent variables. ...
1
vote
0answers
12 views

Why does the computational time of maximum likelihood estimators depend only on the number of UNIQUE observations?

The following paper says that the computational time of least squares, maximum likelihood, and M-estimators in general depend only on the number of UNIQUE observations. Intuitively it makes sense to ...
0
votes
0answers
4 views

Finding expected range of values in multi variable reduction

For a least-squares reduction to find expected values for $a,b,c,d, ...$ from a number of equations like: $a + b = n_1$ $a + c = n_2$ $b + d = n_3$ $c + d = n_4$ ... My question relates to ...
1
vote
1answer
30 views

Linear regression using running parameters

I always asked myself what was the right method name for a simple linear regression using running parameters. I mean that instead of using constant mean $\bar{y}$ or $\bar{x}$ for the estimation of $ ...
0
votes
0answers
10 views

DiD estimator where all groups have some treatment

I am analysing some data on the impact of immigration on peoples attitudes to welfare. I wanted to use a difference in difference estimator as an exogenous shock caused immigration to drastically ...
1
vote
2answers
30 views

What OLS assumptions need to hold when fitting a linear model with machine learning?

I often hear that machine-learning algorithms are not restricted to the assumptions behind the Ordinary Least Squares (OLS) estimator for linear models, e.g.: The conditional mean should be zero. ...
11
votes
6answers
947 views

Linear regression when Y is bounded and discrete

The question is straightforward: Is it appropriate to use linear regression when Y is bounded and discrete (e.g. the test score 1~100, some pre-defined ranking 1~17)? In this case, is it "not good" to ...
2
votes
1answer
21 views

Interaction term and sample selection

I have a dependent variable Y which is continuous. I want to study the impact of X on Y using OLS in a linear model, but I suspect the impact of X is more important for observations with a high value ...
0
votes
0answers
8 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
0
votes
1answer
23 views

OLS regression for not normally distributed data?

In a college course I try to measure the abnormal returns (the returns that are below or over the returns of the market) of a companies stock after a specific event based on linear OLS regression. ...
0
votes
0answers
14 views

Is there a way to know if, when using Least Squares Regression, the line of best fit is over-fitting?

So I am implementing a program that calculates a line of best fit a line segment consisting of 20 pairs of co-ordinates. I know that over-fitting can be a problem but is there a general method using ...
0
votes
0answers
19 views

How are R2 and adjusted R2 mathematically related to the idea of explained variance?

I am trying to understand in what sense, $R^2$ and $R_{adj}^2$ represent the "explained variance." I can't find any similar question that explores the connection in mathematical detail. My current ...
1
vote
2answers
237 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
0
votes
0answers
17 views

Intuition Behind One Vs. All Linear Least Squares Classification

I understand that in the one vs. all classification approach, we form $k$ discriminants, one for each of the $k$ classes and that $(w_k - w_j)^Tx + (b_k - b_j) = 0$ is the hyperplane decision boundary ...