Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
0answers
13 views

VECM model output - where is the long run relationship?

So I'm getting the following EViews output, but where on earth is the long run relationship? Do I have to estimate it separately using OLS? If you have to estimate it yourself via OLS, I've already ...
0
votes
0answers
14 views

When is OLS and FD equal?

I am running an OLS and FD regression. The estimates (on the parameters that appear in both models) are exactly equal, how can this be? By FD I mean: OLS applied to the first difference of the ...
0
votes
0answers
39 views
+50

Assumptions and terminology for dynamic regression with endogenous offset ($y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$)

I'm dealing with a fairly simple time series regression model with the following basic form: $y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$ I'm assuming that observations of $y$ are known without error. $X$ ...
0
votes
0answers
14 views

Applying Dynamic OLS

I am trying to analyze the cointegration relationship between I(1) variables using DOLS and want to check whether the following steps are OK: Select optimal lag of leads and lags using information ...
0
votes
0answers
21 views

Interactions and Multicollinearity

I have a question concerning multicollinearity in OLS. I built a very simple model: y = a + x(x) + y(x) + e. In this case x is a significant predictor, y is not. Both of them are time related ...
0
votes
0answers
14 views

How to find least squares solution with multidimensional data?

I have a data set, on which I want to learn the matrices A and b. So my model is : $$ Y = Ax + b $$ and let's say $x$ is of size $11$x$1$, and $Y$ is $9$x$1$. And I have $50$ observations ($50$ ...
1
vote
0answers
28 views

Stata - regression homework question [closed]

I'm currently attending an econometrics class at my university and it's not my strongest side. So we got a homework problem set and I am having trouble solve it. Here it is: ln E[Y | X1, X2, X3] = ...
1
vote
0answers
10 views

MATLAB: bounding the parameter values in nonlinear modelfitting and AICc scores

I am trying to fit a number of nonlinear models to a dataset, and I need to bound the model parameter values to all be positive. I tried lsqcurvefit function and it works. However, I also need AICc ...
2
votes
0answers
21 views

Prediction Intervals for Incremental OLS regression

I am implementing incremental OLS regression algorithm where the data points arrive one at a time. As the regression parameters are determined by the formula, ...
0
votes
0answers
4 views

Error in enpls.en call [migrated]

I am trying to perform the enpls.en function from the enpls package in R, and I am having a hard time understanding why the function will not commence. ...
1
vote
1answer
25 views

Expectation of squared error

In machine learning, we let $X$ be a real-valued input vector and $Y$ be a real number output, with joint distribution $P(X,Y)$. We are looking for a function $f(X)$ for predicting $Y$ given the ...
3
votes
1answer
41 views

How to calculate the prediction interval for an OLS multiple regression?

What is the algebraic notation to calculate the prediction interval for multiple regression? It sounds silly, but I am having trouble finding a clear algebraic notation of this.
0
votes
0answers
25 views

What is t in ordinary least square?

What is t in this formula? where X is a matrix of independent variables, with the elements of the first column set to 1, and Y is a vector of observations of the dependent variable
2
votes
2answers
103 views

How to change the null hypothesis of the coefficient in the least squares fitting?

I have a least square fitting like this: fit = lsfit(log10(M), log10(RS), wt) This function lists statistics and p-values for the coefficient considering the ...
19
votes
5answers
932 views

Is minimizing squared error equivalent to minimizing absolute error? Why squared error is more popular than the latter?

When we conduct linear regression $y=ax+b$ to fit a bunch of data points $(x_1,y_1),(x_2,y_2),...,(x_n,y_n)$, the classic approach minimizes the squared error. I have long been puzzled by a question ...
1
vote
1answer
36 views

Why can OLS account for non-linearities even though linearity is assumed?

One standard example when introducing OLS in econometric classes is modelling the log-wage by education and experience. Often, the example models account for experience by not only by the experience ...
3
votes
1answer
62 views

Estimation process in OLS with categorical variables and dummy coding

In my question (Cox model on bank customers) regarding the estimation process in regression with categorical variables, @Scortchi write the following: Any coefficient in a multiple regression ...
3
votes
1answer
35 views

Recovering original regression coefficients from standardized

Suppose I use Least Squares to estimate coefficients in the standard linear model with design matrix $X$'s columns standardized, so the model is $$ E[y] = X^*\beta^* $$ where $X^*$ is $X$ with columns ...
1
vote
1answer
32 views

Least-squares training error

In classification problems, the training error typically decreases as further training examples are acquired. However, in my current least-squares problem, the training error actually increases as ...
2
votes
1answer
39 views

Estimation of individual demand for gasoline

Quantity and price of gasoline are clearly endogenous because the quantity and price are determined by the supply and demand. However, the estimation of individual demand for gasoline is often done ...
0
votes
1answer
21 views

Interpreting OLS Cross-Sectional Macro Data

I have the following model: $y=a_0+a_1(x_1/z_1)+a_2(x_2)+e $, where $y$ = the average log of variable y, $x_1$ = the average ratio of $ x_1/z_1 $, $x_2$ = is the averae log of the variable $x_2$. ...
2
votes
0answers
34 views

Adjusting for the past using OLS regression with single lagged response

There are many fine ways to handle a time series error structure in regression, for example as discussed in Time Series with Autoregressive Error. But consider a panel regression model of the form $$ ...
1
vote
1answer
26 views

Mean squared error versus Least squared error, which one to compare datasets?

I have 3 datasets of the same system. But for the first one, I have 21 measurements. For the second and the third one I have only 9 measurements. Now I made a model using these 3 datasets (so 3 ...
0
votes
0answers
23 views

Interpreting interaction terms

I have 9 regional dummies (using only 8 in a regression equation to avoid collinearity, omitted the richest region) and a few explanatory variables such as formal housing (1 if an individual lives in ...
0
votes
1answer
27 views

OLS regression user defined function in Python

Is there a way to handle complex functions for OLS regression in Python? For example, if my function is $y = a - bx^{c} + e^{dx}$, then how I can use a Python library to estimate $a,b,c$ and $d$? i ...
2
votes
3answers
78 views

Estimation of unit-root AR(1) model with OLS

Given a random walk $x_t$, $$x_t=x_{t-1}+\varepsilon_t,$$ consider estimating the slope coefficient $\beta$ in $$x_t=\beta x_{t-1}+\varepsilon_t$$ by OLS. This question and the following answer ...
0
votes
0answers
9 views

Using lag of variable as a proxy to remove endogeneity- Can I just replace the variable in OLS, or do I have to use 2SLS?

I am running a regression of capital, labour and level of migration on GDP (augmented Cobb Douglas production function). To counteract the endogeneity between migration and GDP (migrants might move to ...
1
vote
0answers
30 views

Insignificant squared term but significant linear term

I am estimating the following model: $\ln(y) = \alpha + \beta_1x + \beta_2x^2 $ $\hat\beta_2$ is insignificant while $\hat\beta_1$ is significantly different from zero. However they are jointly ...
2
votes
0answers
29 views

piecewise linear regression with unknown number of knots

I have a model that depends linearly on $v$ and $\alpha$, but not linearly on other two parameters $T_0$ and $T_1$: $f(i; v, \alpha, T_0, T_1)$. Using least squares, I can solve for $v$ and $\alpha$, ...
1
vote
1answer
35 views

Can I use OLS to analyse Cross-sectional data?

I am conducting analysis of cross-sectional data for research purpose, so will it be wise to employ OLS to analyse cross-sectional data and find out the corresponding coefficients of those variables ...
1
vote
0answers
69 views

Can you leave out more than one dummy variable to have more than one variables in the reference category?

For example, in the simple OLS regression: $y = a + b_1x_1 + ... + b_kx_k + \varepsilon$ if your dummy variable $d$ has 10 categories, could you include just one dummy variable for instance: $y ...
1
vote
1answer
40 views

An example of autocorrelation in residuals causing misinterpretation

I'm looking for an example of time series data where a regression of y~x has autocorrelation in the residuals that leads to misinterpreting the model. This is for a class demonstration where I would ...
5
votes
1answer
39 views

Range of lambda in elastic net regression

$\def\l{|\!|}$ Given the elastic net regression $$\min_b \frac{1}{2}\l y - Xb \l^2 + \alpha\lambda \l b\l_2^2 + (1 - \alpha) \lambda \l b\l_1$$ how can an appropriate range of $\lambda$ be chosen ...
0
votes
1answer
57 views

Can someone explain the mechanics of the variance-covariance matrix in OLS?

I have read many of similar posts here already as well as other resources on the topic, but they all generally just show the steps that generate this equation: $\hat{\sigma^2}({X}'X)^{-1}$ What I ...
6
votes
2answers
121 views

Is OLS Asymptotically Efficient Under Heteroscedasticity

I know that OLS is unbiased but not efficient under heteroscedasticity in a linear regression setting. In Wikipedia http://en.wikipedia.org/wiki/Minimum_mean_square_error The MMSE estimator is ...
6
votes
2answers
100 views

multiple local optimum solutions when we solve a linear regression?

I read this statement on one old exam that be (True / False). We can get multiple local optimum solutions if we solve a linear regression problem by minimizing the sum of squared errors using ...
0
votes
0answers
27 views

ols and multiple regression model wth two variables

My model is as follows: $y=b_1*X+b_2*\max(X,0)+u$ Will I have any problems with the variables $X$ and $\max(X,0)$, concerning any correlation issues? Can I just apply the classic OLS methodology?
1
vote
1answer
35 views

Are $\hat{\beta}_{\text{ls}}$ and $S^2$ independent if errors are not normally distributed?

When estimating a linear model $$ Y_i = X_i\beta + \varepsilon_i \quad \quad 1\leq i\leq n$$ We have $\hat{\beta}$ the least squares estimation of the slope and the estimation of the variance, $S^2 = ...
0
votes
1answer
44 views

Need help in understanding continuous-dummy interaction in OLS regression

I am having some conceptual difficulties in understanding and interpreting interaction terms (between a dummy and a continuous variable) in OLS regressions. I was hoping someone could help me out. I ...
5
votes
1answer
67 views

If $\operatorname{Var}\left(\epsilon_i\right) = h\left(X\right) \neq \sigma^2$, what can we know about $\operatorname{Var}\left(\hat{\beta}\right)$?

This question uses the derivations found here. The short version Consider a regression model. If the error variance is a known function of the data (rather than a constant), under what conditions ...
0
votes
0answers
66 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
53 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
0
votes
0answers
18 views

Weighting scheme in ols

I am trying to build a weighted least squares model from some data that I have (x,y,z). y is known to be linear in x, but I have data for different groups of z. So z is a factor variable. The function ...
3
votes
3answers
59 views

Does inconsistent causation mean inconsistent estimator?

I have this problem. I have Y (market share) and X (store size). I want to predict Y from X using a linear regression ... I run OLS to find the betas, their pvalue is meaningful, yada, yada, yada ...
0
votes
1answer
41 views

Preventing overfitting with Least Squares Linear Regression via QR decomposition

I am trying to solve a linear regression problem in an automated fashion, however am having a problem with extremely large weights. I have several thousand datasets, and am running linear regression ...
0
votes
3answers
46 views

Equating the two equations of Ridge Regression

I'm studying Ridge Regression now and I'm having a bit of trouble understanding how to relate the two equations that pop up when I read about it. There is the coefficient estimate: $$\hat{\beta} = ...
0
votes
1answer
34 views

When a slope doesn't match a visual trend

I am trying to determine the linear slope between two variables in a dataset. The ordinary least squares (OLS) method returns a slope which does not appear to fit the trend that one's eye sees in the ...
0
votes
0answers
18 views

What issues may I face when interpolating my dependent variable in an OLS regression?

I'm doing my undergrad dissertation on what host-country factors impact FDI inflows - FDI inflows to the UK is my dependent variable. All of the independent variables I have managed to find at a ...
2
votes
1answer
46 views

$F$-test for hypothesis $\beta_1+\beta_2=2\beta_3$ in a regression

In a regression $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$, how do I use an $F$-test to test the hypothesis $\beta_1+\beta_2=2\beta_3$? The standard $F$-test would test a ...