Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

1
vote
2answers
33 views

Is minimizing squared error equivalent to minimizing absolute error? Why squared error is more popular than the latter?

When we conduct linear regression $y=ax+b$ to fit a bunch of data points $(x_1,y_1),(x_2,y_2),...,(x_n,y_n)$, the classic approach minimizes the squared error. I have long been puzzled by a question ...
1
vote
1answer
30 views

Why can OLS account for non-linearities even though linearity is assumed?

One standard example when introducing OLS in econometric classes is modelling the log-wage by education and experience. Often, the example models account for experience by not only by the experience ...
0
votes
0answers
8 views

Feasible Weighted Least squares (FWLS) variance estimator

Consider the model $y_i = x_i'\beta + \epsilon_i$ where $\operatorname{Var}[\epsilon_i] = \sigma_j^2$, where $j = 1$ for $i=1,\dots m$ and $j=2$ for $i=m+1,\dots n$. Provide the ...
2
votes
1answer
40 views

Estimation process in OLS with categorical variables and dummy coding

In my question (Cox model on bank customers) regarding the estimation process in regression with categorical variables, @Scortchi write the following: Any coefficient in a multiple regression ...
3
votes
1answer
31 views

Recovering original regression coefficients from standardized

Suppose I use Least Squares to estimate coefficients in the standard linear model with design matrix $X$'s columns standardized, so the model is $$ E[y] = X^*\beta^* $$ where $X^*$ is $X$ with columns ...
1
vote
1answer
30 views

Least-squares training error

In classification problems, the training error typically decreases as further training examples are acquired. However, in my current least-squares problem, the training error actually increases as ...
2
votes
1answer
33 views

Estimation of individual demand for gasoline

Quantity and price of gasoline are clearly endogenous because the quantity and price are determined by the supply and demand. However, the estimation of individual demand for gasoline is often done ...
0
votes
1answer
17 views

Interpreting OLS Cross-Sectional Macro Data

I have the following model: $y=a_0+a_1(x_1/z_1)+a_2(x_2)+e $, where $y$ = the average log of variable y, $x_1$ = the average ratio of $ x_1/z_1 $, $x_2$ = is the averae log of the variable $x_2$. ...
2
votes
0answers
32 views

Adjusting for the past using OLS regression with single lagged response

There are many fine ways to handle a time series error structure in regression, for example as discussed in Time Series with Autoregressive Error. But consider a panel regression model of the form $$ ...
1
vote
1answer
23 views

Mean squared error versus Least squared error, which one to compare datasets?

I have 3 datasets of the same system. But for the first one, I have 21 measurements. For the second and the third one I have only 9 measurements. Now I made a model using these 3 datasets (so 3 ...
0
votes
0answers
21 views

Interpreting interaction terms

I have 9 regional dummies (using only 8 in a regression equation to avoid collinearity, omitted the richest region) and a few explanatory variables such as formal housing (1 if an individual lives in ...
0
votes
1answer
25 views

OLS regression user defined function in Python

Is there a way to handle complex functions for OLS regression in Python? For example, if my function is $y = a - bx^{c} + e^{dx}$, then how I can use a Python library to estimate $a,b,c$ and $d$? i ...
2
votes
3answers
75 views

Estimation of unit-root AR(1) model with OLS

Given a random walk $x_t$, $$x_t=x_{t-1}+\varepsilon_t,$$ consider estimating the slope coefficient $\beta$ in $$x_t=\beta x_{t-1}+\varepsilon_t$$ by OLS. This question and the following answer ...
0
votes
0answers
9 views

Using lag of variable as a proxy to remove endogeneity- Can I just replace the variable in OLS, or do I have to use 2SLS?

I am running a regression of capital, labour and level of migration on GDP (augmented Cobb Douglas production function). To counteract the endogeneity between migration and GDP (migrants might move to ...
1
vote
0answers
27 views

Insignificant squared term but significant linear term

I am estimating the following model: $\ln(y) = \alpha + \beta_1x + \beta_2x^2 $ $\hat\beta_2$ is insignificant while $\hat\beta_1$ is significantly different from zero. However they are jointly ...
2
votes
0answers
28 views

piecewise linear regression with unknown number of knots

I have a model that depends linearly on $v$ and $\alpha$, but not linearly on other two parameters $T_0$ and $T_1$: $f(i; v, \alpha, T_0, T_1)$. Using least squares, I can solve for $v$ and $\alpha$, ...
1
vote
1answer
34 views

Can I use OLS to analyse Cross-sectional data?

I am conducting analysis of cross-sectional data for research purpose, so will it be wise to employ OLS to analyse cross-sectional data and find out the corresponding coefficients of those variables ...
1
vote
0answers
68 views

Can you leave out more than one dummy variable to have more than one variables in the reference category?

For example, in the simple OLS regression: $y = a + b_1x_1 + ... + b_kx_k + \varepsilon$ if your dummy variable $d$ has 10 categories, could you include just one dummy variable for instance: $y ...
1
vote
1answer
39 views

An example of autocorrelation in residuals causing misinterpretation

I'm looking for an example of time series data where a regression of y~x has autocorrelation in the residuals that leads to misinterpreting the model. This is for a class demonstration where I would ...
5
votes
1answer
32 views

Range of lambda in elastic net regression

$\def\l{|\!|}$ Given the elastic net regression $$\min_b \frac{1}{2}\l y - Xb \l^2 + \alpha\lambda \l b\l_2^2 + (1 - \alpha) \lambda \l b\l_1$$ how can an appropriate range of $\lambda$ be chosen ...
0
votes
1answer
54 views

Can someone explain the mechanics of the variance-covariance matrix in OLS?

I have read many of similar posts here already as well as other resources on the topic, but they all generally just show the steps that generate this equation: $\hat{\sigma^2}({X}'X)^{-1}$ What I ...
6
votes
2answers
118 views

Is OLS Asymptotically Efficient Under Heteroscedasticity

I know that OLS is unbiased but not efficient under heteroscedasticity in a linear regression setting. In Wikipedia http://en.wikipedia.org/wiki/Minimum_mean_square_error The MMSE estimator is ...
6
votes
2answers
96 views

multiple local optimum solutions when we solve a linear regression?

I read this statement on one old exam that be (True / False). We can get multiple local optimum solutions if we solve a linear regression problem by minimizing the sum of squared errors using ...
0
votes
0answers
27 views

ols and multiple regression model wth two variables

My model is as follows: $y=b_1*X+b_2*\max(X,0)+u$ Will I have any problems with the variables $X$ and $\max(X,0)$, concerning any correlation issues? Can I just apply the classic OLS methodology?
1
vote
1answer
35 views

Are $\hat{\beta}_{\text{ls}}$ and $S^2$ independent if errors are not normally distributed?

When estimating a linear model $$ Y_i = X_i\beta + \varepsilon_i \quad \quad 1\leq i\leq n$$ We have $\hat{\beta}$ the least squares estimation of the slope and the estimation of the variance, $S^2 = ...
0
votes
1answer
43 views

Need help in understanding continuous-dummy interaction in OLS regression

I am having some conceptual difficulties in understanding and interpreting interaction terms (between a dummy and a continuous variable) in OLS regressions. I was hoping someone could help me out. I ...
5
votes
1answer
67 views

If $\operatorname{Var}\left(\epsilon_i\right) = h\left(X\right) \neq \sigma^2$, what can we know about $\operatorname{Var}\left(\hat{\beta}\right)$?

This question uses the derivations found here. The short version Consider a regression model. If the error variance is a known function of the data (rather than a constant), under what conditions ...
0
votes
0answers
64 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
53 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
0
votes
0answers
18 views

Weighting scheme in ols

I am trying to build a weighted least squares model from some data that I have (x,y,z). y is known to be linear in x, but I have data for different groups of z. So z is a factor variable. The function ...
3
votes
3answers
57 views

Does inconsistent causation mean inconsistent estimator?

I have this problem. I have Y (market share) and X (store size). I want to predict Y from X using a linear regression ... I run OLS to find the betas, their pvalue is meaningful, yada, yada, yada ...
0
votes
1answer
39 views

Preventing overfitting with Least Squares Linear Regression via QR decomposition

I am trying to solve a linear regression problem in an automated fashion, however am having a problem with extremely large weights. I have several thousand datasets, and am running linear regression ...
0
votes
3answers
46 views

Equating the two equations of Ridge Regression

I'm studying Ridge Regression now and I'm having a bit of trouble understanding how to relate the two equations that pop up when I read about it. There is the coefficient estimate: $$\hat{\beta} = ...
0
votes
1answer
33 views

When a slope doesn't match a visual trend

I am trying to determine the linear slope between two variables in a dataset. The ordinary least squares (OLS) method returns a slope which does not appear to fit the trend that one's eye sees in the ...
0
votes
0answers
16 views

What issues may I face when interpolating my dependent variable in an OLS regression?

I'm doing my undergrad dissertation on what host-country factors impact FDI inflows - FDI inflows to the UK is my dependent variable. All of the independent variables I have managed to find at a ...
2
votes
1answer
44 views

$F$-test for hypothesis $\beta_1+\beta_2=2\beta_3$ in a regression

In a regression $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$, how do I use an $F$-test to test the hypothesis $\beta_1+\beta_2=2\beta_3$? The standard $F$-test would test a ...
3
votes
0answers
25 views

Adjust linear regression penalty for under/over-estimations

Basically I have a case where under-predictions are worse than over-predictions. Is there a way to penalize the linear regression model during training according to some predefined ratio? E.g. I want ...
3
votes
1answer
42 views

What happens in linear regression if Y's are not independently sampled?

What happens in linear regression $Y\sim X$ when the $Y$'s are not independently sampled and, particularly, may be autocorrelated? I believe the estimator will still work. But what will happen to ...
2
votes
1answer
106 views

Is $Y \sim X$ equivalent to $ln(Y) \sim ln(X)$?

I read in this thread that $Y \sim X$ is equivalent to $ln(Y) \sim ln(X)$ (assuming $X>0$ and without considering standard error issues). Indeed OLS theory says that heteroskedasticity of the ...
0
votes
0answers
15 views

Interaction model with measurement error without replicate measurements

I am currently working on a simple OLS model with two independent variables and one dependent variable: \begin{equation} \ Y_i = \beta_0 + \beta_1 X_i + \beta_2 S_i + \beta_3 X_iZ_i \end{equation} ...
2
votes
1answer
57 views

Nonlinear total least squares / Deming regression in R

I've been using nls() to fit a custom model to my data, but I don't like how the model is fitting and I would like to use an approach that minimizes residuals in ...
0
votes
0answers
60 views

Pooled OLS, fixed, random or mixed effects?

I am analysing a simple balanced panel data with the following variables: ...
2
votes
0answers
24 views

Generating Random Sample to Fit LM Output

I am trying to reverse generate a dataset that led to a certain R lm() output, l I tried to generate random sample like this, and ran lm() ...
0
votes
0answers
10 views

panel data wage equations

I have in my panel data X countries, personal IDs for each individual per country and year. The panel runs for X years. I set the panel based on ID and year. IDs are grouped per country. Running ...
0
votes
0answers
24 views

Specification Error and Bias

True model: $Y = α + βX+ γZ + U$, where U is the error with OLS properties If $X= δ_0 + δ_1Y + δ_2A + δ_3B + R$ , where A and B are exogenous, and R is the error , How can we see that the OLS ...
0
votes
0answers
19 views

regression with autocorrelated errors and specific error structure

I have to fit a linear regression model that takes into account both a specific - conditional variance relationship and a regression form $y_i=a+\beta \times x+\sqrt{\gamma \times x^2}\times ...
12
votes
3answers
143 views

Why trace of $I−X(X′X)^{-1}X′$ is $n-p$ in least square regression when the parameter vector $\beta$ is of p dimensions?

In the model ${y} = X \beta + \epsilon$, we could estimate $\beta$ using the normal equation: $$\hat{\beta} = (X'X)^{-1}X'y,$$ and we could get $$\hat{y} = X \hat{\beta}.$$ The vector of residuals ...
6
votes
5answers
309 views

Recommendation for linear regression with least squares book

I watched several videos on linear regression, mainly from Khan Academy. As I have no background in statistics, I thought this was a good way to get an idea of the topic. However I'm currently writing ...
1
vote
0answers
32 views

Is the sum of all elements of the residual matrix equal zero under OLS?

I have the following OLS model $$ y_i= α+βx_i+ε_i , i = 1,...,N $$ I want to prove that $$ \sum_{k=1}^N\sum_{j=1}^N e_je_k =0$$ I did the following $$ \sum_{k=1}^N (e_1+e_2+e_3+...+e_n) e_k ...