Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

5
votes
1answer
38 views

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
2
votes
1answer
39 views

In OLS is it methodologically correct to use the variance of a variable as an explanatory variable?

Are some OLS assumptions not satisfied if I use the variance of a variable as a proxy of uncertainty in a regression? For instance, would it be methodologically correct if I use moving averages of ...
2
votes
2answers
108 views

Is there a result showing a relation between the size of the residuals and the correlation coefficient?

Consider an OLS regression between two variables. Is there any result which relates the size of the residuals (measured, perhaps, by the sum of the squares) to the Pearson correlation coefficient of ...
0
votes
0answers
20 views

Does this kind of overlap between in-sample data and forecast cause inflated $R^2$?

I am using a simple UIP model to forecast exchange rates using interest rates with a twelve month horizon. The equation I use is: $E(t+12) - E(t) = α + β(I*(t) - I(t))$. I apply OLS linear regression ...
0
votes
0answers
21 views

Is there a way to model the error term in a linear regression with R?

I'm tryin to estimate a pretty basic regression. I have a dataset containing x and y and would like to esimate the following ...
0
votes
0answers
14 views

Proxy variable approach to deal with endogeneity due to reverse causality

Is this a feasible thing to do? I've mostly seen the proxy variable approach used for dealing with omitted variable bias and measurement error. But how would one go about using it for simultaneity? ...
2
votes
2answers
87 views

Regression to solve system of a system of linear equations?

The question I am going to ask probably has a straight-forward answer, I just don't know how to frame my question in a way that would be 'normal.' I have a set of a set of linear equations that I ...
1
vote
0answers
19 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
2
votes
1answer
30 views

Polynomial regression rules

With OLS regression the errors must be normally distributed and be homoscedastic. Does these rules apply to polynomial regression as well?
0
votes
0answers
15 views

Non Significant MANOVA but significant OLS regression

I have data which is showing a $p$-value of $0.15$ significance value within a MANOVA (including a significant planned contrast). However, when I dummy code and put it into an OLS regression, the ...
1
vote
1answer
25 views

Variance of a variable constructed from parameter estimates and predicted values of a linear regression

My data set is an annual panel data set on individual income, the year of unemployment and a number of demographic variables. I run an OLS regression of the form $y_{it} = \sum _{j=1} ^n D_{j,it} ...
4
votes
1answer
140 views

Total least squares curve fit problem

I am trying to fit a quadratic curve across a scatter plot of two variables. Since both variables are noisy I cannot use an ordinary least square regression (OLS) and I would like to have a ...
3
votes
1answer
45 views

If not minimizing SSEs, what am I doing?

I've got a problem that can be summarized as a linear regression. Thus it takes the following form: $$ Y=X \beta +\epsilon $$ where $Y$ and $\epsilon$ are vectors of size $N\times1$, $X$ is a matrix ...
1
vote
1answer
44 views

Endogenous variable and statistical significance in OLS?

I was wondering what the following OLS scenario would imply: a variable is endogenous (i.e. correlated with the error term) yet is statistically significant. Alternatively, what if in, once again ...
1
vote
1answer
71 views

Is this an example of Pooled OLS on Panel Data?

I am looking at a study that analyzes the effect of an infrastructure index on infant mortality and child mortality rates. The database has (asset) quintile level data for 47 different countries (from ...
1
vote
0answers
47 views

choosing $β_0$ and $β_1$ to minimize the residual sum of squares

I'm reading a book called An Introduction to Statistical Learning: with Applications in R, and I have a question in regards to the material inside. I understand that we can find the residual sum of ...
1
vote
1answer
39 views

Using OLS for Model Selection and Prediction - Heteroscedasticity Issue

I am new to regression and having problem in solving Heteroscedasticity in OLS. Have done lots of homework and test before seeking your advice. Sharing the background and what I have done to solve the ...
1
vote
2answers
37 views

Why representation of AR process comes up in estimation

Let ${X_t}$, $t=...-2,-1,0,1,2...$ be a stochastic process that satisfies: $X_t=\rho X_{t-1}+\varepsilon_t$ with $|\rho|<1$ and $\varepsilon_t$ is a white noise. In that case, we also know that ...
2
votes
0answers
21 views

How to approximate error on Chi-Squared Fit when bin counts are zero

I am using a galaxy image simulator that provides a 2D histogram that has the number of photons per pixel (bin) $N$. I am currently using a least-squares residual: $\sum_{bin}(f_{data}-f_{model})^2$ ...
5
votes
3answers
124 views

How to interpret coefficients of $x$ and $x^2$ in same regression

If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two ...
0
votes
1answer
67 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
2answers
105 views

Hat matrix,$H=X(X^{\prime}X )^{-1}X^{\prime}$

What is the importance of hat matrix, $H=X(X^{\prime}X )^{-1}X^{\prime}$ in regression analysis? Is it only for easier calculation ?
3
votes
1answer
84 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
2
votes
1answer
34 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
2
votes
2answers
56 views

Steps in making a Global Regression Model

I saw a journal article [1] saying that he constructed the following S-curve model : $$y=\exp\left({\beta_1 + \frac{\beta_2}{x}}\right) + \mathrm{residual}$$ The topic was about a global regression ...
1
vote
0answers
12 views

How to select an error/weight for two-dimensional binned data?

Beginning with noisy data vectors $\mathbf{x}$ and $\mathbf{y}$, I have binned the data to vectors $\mathbf{x}_b$ and $\mathbf{y}_b$ of length $N_b$ with fixed linear ($\mathbf{x}_b^i - ...
1
vote
1answer
87 views

Generalized Least Squares vs Ordinary Least Squares under a special case

This question regards the problem of Generalized Least Squares. Vectors and matrices will be denoted in bold. Premises. Let $N,K$ be given integers, with $K \gg N > 1$. The transpose of matrix ...
2
votes
1answer
57 views

N-sigma curves for a non-linear least square curve fit

I'm using python's scipy.optimize.curve_fit routine (which uses a non-linear least squares) to fit an exponential function of the form: ...
0
votes
0answers
48 views

Constrained Least Squares/Fixing Data

So I have maybe an unusual question. I have some simulation data that I use as input into a theoretical model. The problem is that the simulated data is noisy and this causes divergences when I try ...
3
votes
1answer
58 views

Weighted least squares

Consider the estimator $b_1=\frac{\sum y_i}{\sum x_i}$. Suppose that $y_i = \beta x_i + \epsilon_i$, $E[\epsilon_i]=0$, $E[\epsilon_i \epsilon_j] (i \neq j)$ and $E[\epsilon_i^2]=\sigma_i^2$. Find ...
0
votes
1answer
31 views

Standard error of difference of estimates

I have two (non-independent) OLS-parameter estimates each with its own standard error. I'm trying to find out what the standard error of the difference of the estimates should be. Can anyone help? Is ...
0
votes
0answers
18 views

Questions about $R^2$, VIFs and very non normal input variables

I have been working with a small part of my dataset trying to eliminate variables and do some micro models. When analysing my micro set I initially found a few high correlations with inputs (0.95 ...
0
votes
0answers
18 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
4
votes
1answer
65 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
1
vote
1answer
57 views

Statsmodels OLS and MSE

So, I have data set and I calculate the model parameters and errors using statsmodels: result = sm.OLS(y, X).fit() result.summary() Now, ...
0
votes
0answers
12 views

Repeated measures with multiple time points for the predictor and dependent variable: Does Xt-1 predict Yt better than Yt-1 predict Xt?

I have a question on what type of analysis I should be looking into to analyze some data I have: Suppose you have 2 runners X and Y, and they take turns sprinting 100 meters, with runner X going ...
0
votes
0answers
15 views

Properties of MLE and least squares methods for estimating parameters of ar(ma) models

I have annual data that seem to have a bimodal density function. My explaination is that there is a distinction between wet and dry years. For my work I would like to use an ar(1)-model for this. ...
0
votes
1answer
34 views

Ordinary least squares - change response and explanatory variable

In a simple regression model using OLS, why is it not, at least in general, possible to move the $x_i$'s to the left-hand side of the model and the $y_i$'s to the right hand side when I want to switch ...
1
vote
3answers
62 views

Correcting data for heteroscedasticity in a regression model

I applied OLS on a regression model that looks as follows: $$ y = b_0 + b_1x_1 + b_2x_2 $$ and found that signs of heteroscedasticity. In an econometrics text book, I found that I can divide each ...
2
votes
0answers
54 views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
2
votes
1answer
126 views

I've found two equations for regression slope, but they give me two different answers. What am I missing?

According to my notes, in a statistical model where $$Y_i=\beta_0 + \beta_1(x) + u$$ (where $u$ is the error term) the predicted slope is $$\hat{\beta}_1 = \frac{\operatorname{Cov}(X, ...
3
votes
0answers
39 views

What is a least angle regression?

Conceptually, I don't understand what is a least angle regression and why it solves LASSO http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf We know that LASSO is $$\min_x||Ax - ...
3
votes
1answer
64 views

How to treat this OLS based on residual diagnostics

I am struggling already a couple of days with this simple OLS, can you help? Outcome years in function of predictor score, very simple linear model. The residual plot does absolutely not look good ...
1
vote
2answers
48 views

Estimation of so called HAR model

Consider an observed time series $\{Y_t\}_{t=1}^T$ and averaged values $$ Y_t^{(h)}=\frac{1}{h} \sum_{i=0}^{h-1} Y_{t-i} $$ and what is called an HAR model (this is a specific example) $$ ...
1
vote
1answer
73 views

Compare two regressions

I try to determine India’s fertility decline between 1991 and 2001 in a multivariate regression (OLS). I have used “total fertility rate” as dependent variable and estimate the effects from six ...
0
votes
0answers
37 views

How to test for autocorrelation with pooled OLS?

Wooldridge 2002 describes how to test for serial correlation in pooled OLS but I don´t get it when I have to use it in STATA. Does anyone know how to test for serial correlation after pooled ols? I ...
2
votes
1answer
83 views

Why is the plot of residuals against fitted values a horinzontal line when the dependent variable is linearly related to the indenpendent variable?

In ordinary least squares regression (OLS), if the plot of the residuals against the fitted values form a horizontal line around 0, then we can say that the dependent variable is linearly related to ...
1
vote
1answer
93 views

How to do Ordinary Least Squares (OLS) when the observations are not linear?

This question arose from this question. Does anyone have some worked examples of an OLS question where the observations are not linear? e.g. $y_i = \alpha + \sin (x_i) + \epsilon_i$ I tried to find ...
2
votes
0answers
47 views

How to detect outliers with longitudinal data?

I am running a pooled OLS and Random Effects (RE) model and I would like to test for whether there are any outliers. I know how to do this for OLS, but I just dont know how to do it for Random ...
1
vote
1answer
48 views

Help clarify the implication of linearity in an Ordinary Least Squares (OLS) Regression

If the dependent variable is linearly related to the independent variables, there should be no systematic relationship between the residuals and the fitted values. In other words, the model should ...