Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

1
vote
1answer
77 views

OLS vs IV estimates - Sign and Significance

Assume I have an equation with 1 endogenous variable, and many other exogenous variables. Also assume I have 2 valid instruments for the endogenous variable for IV estimation. If I were to estimate ...
1
vote
0answers
19 views

Orthogonal projection of a vector which is already orthogonal to part of the basis

Context This question emerged from trying to solve problem 5.1. of Wooldridge, Econometric Analysis of Cross Section and Panel Data. The problem asks to show the equivalence of the estimators ...
0
votes
1answer
79 views

How can you prove that the naive estimator is less efficient than the OLS estimator

The "naive estimator" is an estimate of the slope obtained by joining the first and last observations and dividing the increase in the height by the horizontal distance between them. Given that the ...
0
votes
0answers
10 views

Arbitrary least squares fitting with an initial model

I have a problem that seems like a straightforward implementation of linear least-squares, but I can't quite figure out how it will work. I have a (noisy) dataset $f(x,y)$ where $f$ is is non-linear ...
1
vote
1answer
43 views

Regressing a differenced variable on a lagged variable. How can I fix the error in R?

I have a time series (std) of 324 observations with no missing values, starting from January 1987 and ending in December 2013. I want to regress via OLS the one in the question. In R, the code: ...
0
votes
1answer
18 views

Why do we use fixed effects and controls at the same time?

When we run a regression model (say OLS for simplicity) of y ~ x, we might have to use several control variables say z1 and z2. Now our model is y ~ x + z1 + z2, we may believe that z1 and z2 are not ...
1
vote
0answers
27 views

How much does the value of a car depreciate due to time (only), and mileage (only)?

I have a database of second hand cars. It contains (among other things) the asking price, the mileage, the fuel consumption and the year it was built. I would like to know how much the value ...
1
vote
1answer
42 views

Regression model result interpretation

I am trying to learn few things about generalized least square regression modeling. Here are the models that I am using., X1 ~ Y1 + Y2, X2 ~ Y1 + Y4, All predictor variables, Y1, Y2 and Y4 are ...
3
votes
1answer
33 views

Prediction error in least squares with a linear model

In the classical linear model with $$Y=X\beta +\epsilon,$$ where $Y \in \mathbb{R}^n$ is the observation, $X\in \mathbb{R}^{n\times p}$ is the known covariates, $\beta \in \mathbb{R}^p$ is the ...
1
vote
2answers
59 views

Does least squares regression imply normality of errors?

For the linear model $$y_i=\beta_0 +\sum_{k=1}^{n}\beta_k x_{ik} + \epsilon_i$$ the parameter estimates are the same for the maximum likelihood method and the least square method (minimizing ...
1
vote
2answers
51 views

What is the effect of having skewed dependent variable on scatterplot result

The histogram of my dependent variable is as following: I draw the scatter plot of my dependent variable and independent variable, and the result is as following picture? I am wondering if the ...
3
votes
2answers
47 views

what is difference between ordinary least squares and residuals

I think the ordinary least squares are the sum of the vertical distance between the observed data to the model line(regression). And residual is calculated by add up all vertical distance between each ...
5
votes
1answer
83 views

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
2
votes
1answer
39 views

In OLS is it methodologically correct to use the variance of a variable as an explanatory variable?

Are some OLS assumptions not satisfied if I use the variance of a variable as a proxy of uncertainty in a regression? For instance, would it be methodologically correct if I use moving averages of ...
2
votes
2answers
111 views

Is there a result showing a relation between the size of the residuals and the correlation coefficient?

Consider an OLS regression between two variables. Is there any result which relates the size of the residuals (measured, perhaps, by the sum of the squares) to the Pearson correlation coefficient of ...
0
votes
0answers
21 views

Does this kind of overlap between in-sample data and forecast cause inflated $R^2$?

I am using a simple UIP model to forecast exchange rates using interest rates with a twelve month horizon. The equation I use is: $E(t+12) - E(t) = α + β(I*(t) - I(t))$. I apply OLS linear regression ...
0
votes
0answers
24 views

Is there a way to model the error term in a linear regression with R?

I'm tryin to estimate a pretty basic regression. I have a dataset containing x and y and would like to esimate the following ...
0
votes
0answers
15 views

Proxy variable approach to deal with endogeneity due to reverse causality

Is this a feasible thing to do? I've mostly seen the proxy variable approach used for dealing with omitted variable bias and measurement error. But how would one go about using it for simultaneity? ...
2
votes
2answers
89 views

Regression to solve system of a system of linear equations?

The question I am going to ask probably has a straight-forward answer, I just don't know how to frame my question in a way that would be 'normal.' I have a set of a set of linear equations that I ...
1
vote
1answer
28 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
2
votes
1answer
31 views

Polynomial regression rules

With OLS regression the errors must be normally distributed and be homoscedastic. Does these rules apply to polynomial regression as well?
0
votes
0answers
15 views

Non Significant MANOVA but significant OLS regression

I have data which is showing a $p$-value of $0.15$ significance value within a MANOVA (including a significant planned contrast). However, when I dummy code and put it into an OLS regression, the ...
1
vote
1answer
27 views

Variance of a variable constructed from parameter estimates and predicted values of a linear regression

My data set is an annual panel data set on individual income, the year of unemployment and a number of demographic variables. I run an OLS regression of the form $y_{it} = \sum _{j=1} ^n D_{j,it} ...
4
votes
1answer
166 views

Total least squares curve fit problem

I am trying to fit a quadratic curve across a scatter plot of two variables. Since both variables are noisy I cannot use an ordinary least square regression (OLS) and I would like to have a ...
3
votes
1answer
45 views

If not minimizing SSEs, what am I doing?

I've got a problem that can be summarized as a linear regression. Thus it takes the following form: $$ Y=X \beta +\epsilon $$ where $Y$ and $\epsilon$ are vectors of size $N\times1$, $X$ is a matrix ...
1
vote
1answer
53 views

Endogenous variable and statistical significance in OLS?

I was wondering what the following OLS scenario would imply: a variable is endogenous (i.e. correlated with the error term) yet is statistically significant. Alternatively, what if in, once again ...
1
vote
1answer
87 views

Is this an example of Pooled OLS on Panel Data?

I am looking at a study that analyzes the effect of an infrastructure index on infant mortality and child mortality rates. The database has (asset) quintile level data for 47 different countries (from ...
1
vote
0answers
47 views

choosing $β_0$ and $β_1$ to minimize the residual sum of squares

I'm reading a book called An Introduction to Statistical Learning: with Applications in R, and I have a question in regards to the material inside. I understand that we can find the residual sum of ...
1
vote
1answer
43 views

Using OLS for Model Selection and Prediction - Heteroscedasticity Issue

I am new to regression and having problem in solving Heteroscedasticity in OLS. Have done lots of homework and test before seeking your advice. Sharing the background and what I have done to solve the ...
1
vote
2answers
37 views

Why representation of AR process comes up in estimation

Let ${X_t}$, $t=...-2,-1,0,1,2...$ be a stochastic process that satisfies: $X_t=\rho X_{t-1}+\varepsilon_t$ with $|\rho|<1$ and $\varepsilon_t$ is a white noise. In that case, we also know that ...
2
votes
0answers
24 views

How to approximate error on Chi-Squared Fit when bin counts are zero

I am using a galaxy image simulator that provides a 2D histogram that has the number of photons per pixel (bin) $N$. I am currently using a least-squares residual: $\sum_{bin}(f_{data}-f_{model})^2$ ...
5
votes
3answers
147 views

How to interpret coefficients of $x$ and $x^2$ in same regression

If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two ...
0
votes
1answer
71 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
2answers
114 views

Hat matrix,$H=X(X^{\prime}X )^{-1}X^{\prime}$

What is the importance of hat matrix, $H=X(X^{\prime}X )^{-1}X^{\prime}$ in regression analysis? Is it only for easier calculation ?
3
votes
1answer
95 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
2
votes
1answer
37 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
2
votes
2answers
58 views

Steps in making a Global Regression Model

I saw a journal article [1] saying that he constructed the following S-curve model : $$y=\exp\left({\beta_1 + \frac{\beta_2}{x}}\right) + \mathrm{residual}$$ The topic was about a global regression ...
1
vote
0answers
14 views

How to select an error/weight for two-dimensional binned data?

Beginning with noisy data vectors $\mathbf{x}$ and $\mathbf{y}$, I have binned the data to vectors $\mathbf{x}_b$ and $\mathbf{y}_b$ of length $N_b$ with fixed linear ($\mathbf{x}_b^i - ...
1
vote
1answer
94 views

Generalized Least Squares vs Ordinary Least Squares under a special case

This question regards the problem of Generalized Least Squares. Vectors and matrices will be denoted in bold. Premises. Let $N,K$ be given integers, with $K \gg N > 1$. The transpose of matrix ...
2
votes
1answer
65 views

N-sigma curves for a non-linear least square curve fit

I'm using python's scipy.optimize.curve_fit routine (which uses a non-linear least squares) to fit an exponential function of the form: ...
0
votes
0answers
49 views

Constrained Least Squares/Fixing Data

So I have maybe an unusual question. I have some simulation data that I use as input into a theoretical model. The problem is that the simulated data is noisy and this causes divergences when I try ...
3
votes
1answer
59 views

Weighted least squares

Consider the estimator $b_1=\frac{\sum y_i}{\sum x_i}$. Suppose that $y_i = \beta x_i + \epsilon_i$, $E[\epsilon_i]=0$, $E[\epsilon_i \epsilon_j] (i \neq j)$ and $E[\epsilon_i^2]=\sigma_i^2$. Find ...
0
votes
1answer
33 views

Standard error of difference of estimates

I have two (non-independent) OLS-parameter estimates each with its own standard error. I'm trying to find out what the standard error of the difference of the estimates should be. Can anyone help? Is ...
0
votes
0answers
18 views

Questions about $R^2$, VIFs and very non normal input variables

I have been working with a small part of my dataset trying to eliminate variables and do some micro models. When analysing my micro set I initially found a few high correlations with inputs (0.95 ...
0
votes
0answers
21 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
4
votes
1answer
72 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
1
vote
1answer
70 views

Statsmodels OLS and MSE

So, I have data set and I calculate the model parameters and errors using statsmodels: result = sm.OLS(y, X).fit() result.summary() Now, ...
0
votes
0answers
13 views

Repeated measures with multiple time points for the predictor and dependent variable: Does Xt-1 predict Yt better than Yt-1 predict Xt?

I have a question on what type of analysis I should be looking into to analyze some data I have: Suppose you have 2 runners X and Y, and they take turns sprinting 100 meters, with runner X going ...
0
votes
0answers
16 views

Properties of MLE and least squares methods for estimating parameters of ar(ma) models

I have annual data that seem to have a bimodal density function. My explaination is that there is a distinction between wet and dry years. For my work I would like to use an ar(1)-model for this. ...
0
votes
1answer
34 views

Ordinary least squares - change response and explanatory variable

In a simple regression model using OLS, why is it not, at least in general, possible to move the $x_i$'s to the left-hand side of the model and the $y_i$'s to the right hand side when I want to switch ...