Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

-1
votes
0answers
12 views

Normal Distribution Stata Help [on hold]

I am running the following regression in Stata ...
0
votes
0answers
17 views

Error in drawing normal distribution from Stata [on hold]

I have to draw a normal distribution for an OLS estimator I am using the command drawnorm betaest, n(10000) means(beta_mean) sds(betasq) However, it is giving me the error no; data in memory ...
0
votes
0answers
24 views

Consistency of variance estimator in OLS [duplicate]

Given the model, $$ y_i = x_i'\beta + \epsilon_i \quad \epsilon_i \sim N(0, \sigma^2) \quad iid \quad \forall i = 1, ..,n $$ how can I prove that the estimator of the variance $\hat{\sigma}^2 = \...
1
vote
1answer
13 views

Upwardly biased forecast results due to period of high demand: how to deal with this?

I'm currently working on a call-center forecasting project with some data limitations. Currently it is still a learning-project, and I started with simple OLS regressions. For the months 2016-12 to ...
1
vote
0answers
17 views

How I can sketch the proof of consistency of only one beta in multiple regression?

Now assume you additionally obtained data on average parental incomes (PI) and the ethnic composition (EC) of the pupils in school. You regress the score on STR PI EC and a constant. State the ...
2
votes
1answer
24 views

Conditions in which the sign of a coefficient will change between a linear probability model and a logistic model

I am estimating a model where the DV is a binary variable and the key independent variable is the interaction between a dummy variable and a continuous variable. I am getting an very odd result where ...
1
vote
0answers
17 views

how to prove B1 as a consistent estimator in panel data

$Yi=a+ B_1*X_i+ B_2*Z_i+\epsilon_i$, and suppose that $Zi$ is unobservable and not correlated with $X_i$. Is the OLS estimator of $B_1$ consistent by regressing $Y_i$ on a constant a and $X_i$? I ...
1
vote
0answers
8 views

How can I implement this Robust PCA equation in a more efficient way?

I recently learned in class the Principle Component Analysis method aims to approximate a matrix X to a multiplication of two matrices Z*W. If X is a n x d matrix, Z is a n x k matrix and W is a k x d ...
0
votes
0answers
9 views

Regression using previous years results

Is it possible to make a linear model non auto regressive using previous years observation's. For example I have a student score, and I have 3 contributing variables (subject scores e.g. science......
1
vote
0answers
22 views

Second order gradient-based method to locally maximize sum of squares

Non-linear least squares algorithms such as Gauss-Newton allow me to (locally) minimize a sum of squares of residuals (the output of some non-linear function). I.e. locally solve: $$ \mathbf{x} = \arg\...
0
votes
0answers
18 views

How to prove that conditional distribution for Y given OLS for simple linear regression do not depends on original parameters? [duplicate]

How to prove that for a simple linear regression model: $$y_i=\beta_0+\beta_1 x_i+\varepsilon_i,$$ the conditional distribution $$Y|\hat{\beta}_0,\hat{\beta}_1$$ do not depends on $\beta_0$ and $\...
1
vote
1answer
22 views

Loss function for KNN Regressor

What is the Loss function for KNN Regressor? Would it be similar to OLS? If so what would be the main difference?
0
votes
0answers
11 views

OLS Multicolleniarity

I have a pretty simple task to estimate ols multiple regression. I need a measure of multicolleniarity. Is condition number a good measure and what criteria exists fot its value?
1
vote
3answers
36 views

Question Regarding Zero Conditional Mean

Hi I am a beginner to econometrics! I have been dealing with bivariate regression. We use the formula $y = \beta_0 + \beta_1 x$. I am told that if $E(u\mid x) \ne 0$ then the estimate of the slope ...
0
votes
0answers
20 views

Difference among MLE, GMM and OLS [on hold]

I want to know what exactly the differences among OLS, GMM and MLE estimators and why statisticians prefer more to implement GMM while econometricians are more likely to use OLS or GMM. Since it is ...
1
vote
1answer
36 views

A very basic question on ENDOGENEITY

In the regression model $Y$ = $\beta_0$ + $\beta_1X_1$ + $\beta_2X_2$ +.......+ $\beta_kX_k$ + $\epsilon$ where $\epsilon$ = $\delta_0X_2$ + $\lambda$ Will this also be the case of endogeneity ...
0
votes
0answers
14 views

Generalized Least Squares and Heteroskedasticity

I am trying to model a OLS where I know that the hetero-skedasticity is like this $E(\epsilon^2)$ = $\sigma_i^2$ = $\delta_0$ + $\delta_1*X_{i2}$ So, I was using the concept of feasible generalized ...
0
votes
0answers
59 views

auto-correlation and OLS regression

I was trying to find the OLS estimator for the model: $Y$ = $\beta_0$ + $\beta_1X_{1t}$ + $\beta_2X_{2t}$ +.......+ $\beta_5X_{5t}$ + $e$ t = 1,2,3 ......, 50 time ordered observations X is a full ...
0
votes
0answers
6 views

Least square estimate for post-stratification sampling

I figured out via the normal linear regression method that Beta0 hat = ybar - Beta1 hat xbar. But I am not sure how to find out the least square estimate for Chat. Is anyone able to help me? Thanks!! ...
1
vote
1answer
33 views

Implications of i.i.d. sample

I have the following question: I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as $\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
23 views

Time as an independent variable in multiple regression

Can we chose time (t) as one of the independent variables in multiple (OLS) regression?
1
vote
1answer
25 views

2-step linear regression

Very simple problem: First model: I run a linear regression of $Y$ on $X$ and $Z$. Second model: I regress $Y$ on $X$ only, compute the residuals, and regress these residuals on $Z$. Why do I ...
0
votes
0answers
9 views

Autocorrelated independent variable (how to treat)

I have some response variable $Y$ that I am trying to model with OLS. There is a single independent variable $X_{t,N}=Y_t-Y_{t-N}$ for some $N$. However, when I do such a simple regression, I obtain ...
0
votes
0answers
21 views

OLS Characteristics - Proof

I am trying to figure out the mathematical proof of one of the OLS characteristics in simple linear regression. One stage, or more accurately, one term is not working out. The term coloured in red is ...
1
vote
1answer
54 views

OLS estimation of intercept in AR($p$) in R

I investigate the performance of the OLS estimator of an AR($3$) model given by $$ X_t=\mu+\phi_1X_{t-1}+\phi_2X_{t-2}+\phi_3X_{t-3}+\varepsilon_t $$ for $t\in\mathbb Z$ using the following code: <...
0
votes
1answer
29 views

Least squares regression coefficient with minimal information

If I only have a correlation matrix of 4 variables and the sample size, is it possible to predict 1 variables from the other 3 while using information about sample size? I’m trying to use lm but my ...
0
votes
0answers
19 views

relation between OLS regressions using different data transformations

I have a $(n \times d)$ panel $y$ of $n$ different variables , and a $(n \times d)$ panel $x$ of their forecasts. $d=$ time length of data $n=$ cross section width/ no. of variables I run a pooled (...
1
vote
0answers
12 views

Why does the computational time of maximum likelihood estimators depend only on the number of UNIQUE observations?

The following paper says that the computational time of least squares, maximum likelihood, and M-estimators in general depend only on the number of UNIQUE observations. Intuitively it makes sense to ...
0
votes
0answers
4 views

Finding expected range of values in multi variable reduction

For a least-squares reduction to find expected values for $a,b,c,d, ...$ from a number of equations like: $a + b = n_1$ $a + c = n_2$ $b + d = n_3$ $c + d = n_4$ ... My question relates to ...
0
votes
0answers
10 views

DiD estimator where all groups have some treatment

I am analysing some data on the impact of immigration on peoples attitudes to welfare. I wanted to use a difference in difference estimator as an exogenous shock caused immigration to drastically ...
11
votes
6answers
948 views

Linear regression when Y is bounded and discrete

The question is straightforward: Is it appropriate to use linear regression when Y is bounded and discrete (e.g. the test score 1~100, some pre-defined ranking 1~17)? In this case, is it "not good" to ...
2
votes
1answer
21 views

Interaction term and sample selection

I have a dependent variable Y which is continuous. I want to study the impact of X on Y using OLS in a linear model, but I suspect the impact of X is more important for observations with a high value ...
0
votes
0answers
8 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
1
vote
2answers
30 views

What OLS assumptions need to hold when fitting a linear model with machine learning?

I often hear that machine-learning algorithms are not restricted to the assumptions behind the Ordinary Least Squares (OLS) estimator for linear models, e.g.: The conditional mean should be zero. ...
0
votes
1answer
23 views

OLS regression for not normally distributed data?

In a college course I try to measure the abnormal returns (the returns that are below or over the returns of the market) of a companies stock after a specific event based on linear OLS regression. ...
0
votes
0answers
14 views

Is there a way to know if, when using Least Squares Regression, the line of best fit is over-fitting?

So I am implementing a program that calculates a line of best fit a line segment consisting of 20 pairs of co-ordinates. I know that over-fitting can be a problem but is there a general method using ...
0
votes
0answers
19 views

How are R2 and adjusted R2 mathematically related to the idea of explained variance?

I am trying to understand in what sense, $R^2$ and $R_{adj}^2$ represent the "explained variance." I can't find any similar question that explores the connection in mathematical detail. My current ...
0
votes
0answers
17 views

Intuition Behind One Vs. All Linear Least Squares Classification

I understand that in the one vs. all classification approach, we form $k$ discriminants, one for each of the $k$ classes and that $(w_k - w_j)^Tx + (b_k - b_j) = 0$ is the hyperplane decision boundary ...
0
votes
0answers
6 views

unbalanced due to regimes?

I have a sample which seems to have data in two distinct regimes. Suppose that 50% of observations have x variable from 0 to 1, while the remaining 50% with x between 1 and 8. Y appears to increase ...
1
vote
0answers
30 views

Spherical error variance in OLS estimation of AR($p$)

Consider the linear model $\boldsymbol y=\boldsymbol X\beta+\boldsymbol\varepsilon$. One of the assumptions for the OLS estimator is the spherical error variance assumption which states that $\...
1
vote
0answers
20 views

Limitations to using the principal of least squares to fit an ellipse to experimental data?

I have used the method described below many times without issue to determine the major and minor axes of the "data ellipse". However, for this set of data (Fig. A) the method described below is ...
1
vote
1answer
30 views

Linear regression using running parameters

I always asked myself what was the right method name for a simple linear regression using running parameters. I mean that instead of using constant mean $\bar{y}$ or $\bar{x}$ for the estimation of $ ...
1
vote
0answers
12 views

Comparison effects of regressors for several dependent variables

I want to study the effect of two explanatory variables $X$ and $Z$ on four different outcomes (which are binary variables, i.e. equal 1 if satisfied and 0 if disatisfied): $Y_1$, $Y_2$, $Y_3$ and $...
0
votes
0answers
8 views

Panel with four dimensions and fixed effects

Let's assume I have a dependent variable $Y$ at the firm level ($i$) that varies across country ($c$), sector ($s$) and time ($t$). It can be written: $Y_{csi,t}$, which means the variables varies ...
0
votes
0answers
13 views

how least square estimation can be done for a distribution

As i have estimated parameters of geometric distribution by using MLE (maximum likelihood estimation) and MOM( Method of moment) but i have problem in estimating parameter of Geometric distribution ...
4
votes
2answers
69 views

Clustered standard errors - Why are SE smaller or bigger than OLS depending on cluster level?

I am analyzing some data using an OLS model. Data represent managers working for US cities. Within each city, we surveyed more than one manager (max 5). Multiple cities per state were surveyed. I'd ...
3
votes
1answer
32 views

Least Squares Estimator Vs Ordinary Least Squares Estimator

Is there any difference between the two terms "Least Squares Estimator" and "Ordinary Least Squares Estimator". For convenience, I'll refer to them as LS and OLS respectively. I have read various ...
1
vote
0answers
43 views

Linear Regression: Calculating a treatment effect directly in regression vs. averaging potential outcomes

Suppose I have the following true model, where an individual $i$ at a particular point in time $t$ is either treated ($W=1)$ or untreated ($W=0$). The outcome for individual $i$ at time $t$ under ...
0
votes
1answer
31 views

Closed-form solution to least squares with a matrix of parameters?

I'm familiar with the closed-form solution of ordinary least squares which minimizes $\sum_{n=1}^N(y_n - \mathbf{\beta x_n})^2$ for scalar $y_n$. However, in my situation I am trying to fit some data ...
1
vote
1answer
37 views

Autocorrelation in residuals

Hi guys, thanks for your time! Problem description: I am working with dynamic factors. I have 4 panels of 24 series (hourly electricity prices) and I reduce them to 1 dynamic factor each that I then ...