Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

4
votes
3answers
107 views

How to interpret coefficients of $x$ and $x^2$ in same regression

If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two ...
0
votes
1answer
59 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
2answers
90 views

Hat matrix,$H=X(X^{\prime}X )^{-1}X^{\prime}$

What is the importance of hat matrix, $H=X(X^{\prime}X )^{-1}X^{\prime}$ in regression analysis? Is it only for easier calculation ?
3
votes
1answer
61 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
0
votes
0answers
26 views

Why are the standard errors of OLS larger here (compared to ML)? [closed]

The data here are obtained from a simulated normal distribution. I know that in case of normality of the disturbances of the Data Generating Process (DGP) OLS is UMVUE. So then why are the standard ...
2
votes
1answer
29 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
2
votes
2answers
40 views

Steps in making a Global Regression Model

I saw a journal article [1] saying that he constructed the following S-curve model : $$y=\exp\left({\beta_1 + \frac{\beta_2}{x}}\right) + \mathrm{residual}$$ The topic was about a global regression ...
1
vote
0answers
8 views

How to select an error/weight for two-dimensional binned data?

Beginning with noisy data vectors $\mathbf{x}$ and $\mathbf{y}$, I have binned the data to vectors $\mathbf{x}_b$ and $\mathbf{y}_b$ of length $N_b$ with fixed linear ($\mathbf{x}_b^i - ...
1
vote
1answer
80 views

Generalized Least Squares vs Ordinary Least Squares under a special case

This question regards the problem of Generalized Least Squares. Vectors and matrices will be denoted in bold. Premises. Let $N,K$ be given integers, with $K \gg N > 1$. The transpose of matrix ...
2
votes
1answer
47 views

N-sigma curves for a non-linear least square curve fit

I'm using python's scipy.optimize.curve_fit routine (which uses a non-linear least squares) to fit an exponential function of the form: ...
0
votes
0answers
44 views

Constrained Least Squares/Fixing Data

So I have maybe an unusual question. I have some simulation data that I use as input into a theoretical model. The problem is that the simulated data is noisy and this causes divergences when I try ...
3
votes
1answer
54 views

Weighted least squares

Consider the estimator $b_1=\frac{\sum y_i}{\sum x_i}$. Suppose that $y_i = \beta x_i + \epsilon_i$, $E[\epsilon_i]=0$, $E[\epsilon_i \epsilon_j] (i \neq j)$ and $E[\epsilon_i^2]=\sigma_i^2$. Find ...
0
votes
1answer
29 views

Standard error of difference of estimates

I have two (non-independent) OLS-parameter estimates each with its own standard error. I'm trying to find out what the standard error of the difference of the estimates should be. Can anyone help? Is ...
0
votes
0answers
13 views

Questions about $R^2$, VIFs and very non normal input variables

I have been working with a small part of my dataset trying to eliminate variables and do some micro models. When analysing my micro set I initially found a few high correlations with inputs (0.95 ...
0
votes
0answers
15 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
4
votes
1answer
59 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
1
vote
1answer
40 views

Statsmodels OLS and MSE

So, I have data set and I calculate the model parameters and errors using statsmodels: result = sm.OLS(y, X).fit() result.summary() Now, ...
0
votes
0answers
6 views

Repeated measures with multiple time points for the predictor and dependent variable: Does Xt-1 predict Yt better than Yt-1 predict Xt?

I have a question on what type of analysis I should be looking into to analyze some data I have: Suppose you have 2 runners X and Y, and they take turns sprinting 100 meters, with runner X going ...
0
votes
0answers
10 views

Properties of MLE and least squares methods for estimating parameters of ar(ma) models

I have annual data that seem to have a bimodal density function. My explaination is that there is a distinction between wet and dry years. For my work I would like to use an ar(1)-model for this. ...
0
votes
1answer
32 views

Ordinary least squares - change response and explanatory variable

In a simple regression model using OLS, why is it not, at least in general, possible to move the $x_i$'s to the left-hand side of the model and the $y_i$'s to the right hand side when I want to switch ...
1
vote
3answers
56 views

Correcting data for heteroscedasticity in a regression model

I applied OLS on a regression model that looks as follows: $$ y = b_0 + b_1x_1 + b_2x_2 $$ and found that signs of heteroscedasticity. In an econometrics text book, I found that I can divide each ...
2
votes
0answers
47 views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
2
votes
1answer
124 views

I've found two equations for regression slope, but they give me two different answers. What am I missing?

According to my notes, in a statistical model where $$Y_i=\beta_0 + \beta_1(x) + u$$ (where $u$ is the error term) the predicted slope is $$\hat{\beta}_1 = \frac{\operatorname{Cov}(X, ...
3
votes
0answers
38 views

What is a least angle regression?

Conceptually, I don't understand what is a least angle regression and why it solves LASSO http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf We know that LASSO is $$\min_x||Ax - ...
3
votes
1answer
61 views

How to treat this OLS based on residual diagnostics

I am struggling already a couple of days with this simple OLS, can you help? Outcome years in function of predictor score, very simple linear model. The residual plot does absolutely not look good ...
1
vote
2answers
41 views

Estimation of so called HAR model

Consider an observed time series $\{Y_t\}_{t=1}^T$ and averaged values $$ Y_t^{(h)}=\frac{1}{h} \sum_{i=0}^{h-1} Y_{t-i} $$ and what is called an HAR model (this is a specific example) $$ ...
1
vote
1answer
62 views

Compare two regressions

I try to determine India’s fertility decline between 1991 and 2001 in a multivariate regression (OLS). I have used “total fertility rate” as dependent variable and estimate the effects from six ...
0
votes
0answers
29 views

How to test for autocorrelation with pooled OLS?

Wooldridge 2002 describes how to test for serial correlation in pooled OLS but I don´t get it when I have to use it in STATA. Does anyone know how to test for serial correlation after pooled ols? I ...
2
votes
1answer
67 views

Why is the plot of residuals against fitted values a horinzontal line when the dependent variable is linearly related to the indenpendent variable?

In ordinary least squares regression (OLS), if the plot of the residuals against the fitted values form a horizontal line around 0, then we can say that the dependent variable is linearly related to ...
1
vote
1answer
91 views

How to do Ordinary Least Squares (OLS) when the observations are not linear?

This question arose from this question. Does anyone have some worked examples of an OLS question where the observations are not linear? e.g. $y_i = \alpha + \sin (x_i) + \epsilon_i$ I tried to find ...
2
votes
0answers
42 views

How to detect outliers with longitudinal data?

I am running a pooled OLS and Random Effects (RE) model and I would like to test for whether there are any outliers. I know how to do this for OLS, but I just dont know how to do it for Random ...
1
vote
1answer
48 views

Help clarify the implication of linearity in an Ordinary Least Squares (OLS) Regression

If the dependent variable is linearly related to the independent variables, there should be no systematic relationship between the residuals and the fitted values. In other words, the model should ...
0
votes
2answers
45 views

Algorithms for prediction of consumption, based on previous data

I have data for a few stores which sell apples. For each store I have an averaged value of how many kilos of apples users have bought per day for this month. So it looks like this: ...
0
votes
1answer
54 views

First order condition of sum of squares with respect to variance of residuals

Consider the criterion function for ordinary least squares $$ S(b)=(Y-X'b)'(Y-X'b) $$ with Y, a matrix of dependent variables, and X, a matrix of explanatory variables. It is of course known that: ...
2
votes
1answer
39 views

Help clarify the implication of normality in an Ordinary Least Square (OLS) Regression

If the dependent variable is normally distributed for a fixed set of predictor values, then the residual values should be normally distributed with a mean of 0. I have two questions based on the ...
0
votes
1answer
47 views

lsqnonneg – variation between results in R and MATLAB

We are trying to do nonlinear least squares fitting in R. We have reference code in MATLAB as below. ...
6
votes
1answer
112 views

Gaussian mixture regression in higher dimensions

Problem: I have a discrete representation of a surface/height-map $z = f(x,y)$ that i want to model as a mixture of gaussians (please take probability distributions out of your mind for a moment). ...
0
votes
1answer
52 views

AR process with a constant

I am having trouble understanding the estimation of an AR process. In some textbooks, the AR(1) process is defined as follows: $y_{t}=\theta y_{t-1}+ϵ_t$ (which does not contain a constant). So the ...
0
votes
2answers
26 views

OLS Coefficient estimator; Transformation from Matrix to sum of matrices form

I do not understand why the following equality holds (taken from Cameron & Trivedi 2005: Microeconomtrics): ...
1
vote
0answers
31 views

Least squares with multiple constraints

I work with a regression with ARMA errors and I want to use LASSO to shrink the coefficients and select my variables. This topic is discussed in the article Wu et al. (2012). So, the problem I have to ...
2
votes
2answers
56 views

OLS results dependent on scaling of independent variable

I have the following python dataframe (5 rows out of nearly 15,000): ...
2
votes
1answer
26 views

How to minimize SSE with given slope of 1?

With given data, I need a line of slope 1 with minimized SSE. Anyone know how I could do this in R or Excel or another program? If you could tell me how to do it mathematically that might help. ...
0
votes
0answers
20 views

Same coefficients for OLS and Full Information Maximum Likelihood

I have an incomplete dataset and default setting in Eviews is listwise deletion. Running my regression (regressing a continuous variable on 22 dummy variable regressors, unfortunately there is no ...
2
votes
0answers
25 views

multicollinearity in OLS regression [duplicate]

If I have a dependent variable Y and two independent variables X1 and X2 , that are highly correlated. Y ~ beta1*X1 + beta2*X2 What issues can multicollinearity cause in an OLS regression, apart ...
1
vote
1answer
31 views

Seemingly unrelated regression and OLS

It's been said that SUR is equivalent to the equation-by-equation OLS: i. When there are no cross-equation correlations between the error terms. ii. When each equation contains exactly the same ...
1
vote
0answers
29 views

Upper Bound of Measured Data

HW question I am having trouble on: My attempt at solving it: I thought part (a) was pretty straight forward until I worked it out and got the same value of K for K_upperbound which I assume is ...
2
votes
0answers
42 views

When is OLS estimation bias due to endogeneity negligible?

I have a question about OLS estimation bias due to endogeneity. Say I have the following panel regression model for the price of good 1 for a particular sale $k$ at time $t$: $$P_{kt}^1=\beta M_{t} + ...
0
votes
2answers
257 views

Multivariate regression with weighted least squares in python?

I have a multivariate regression problem that I need to solve using the weighted least squares method. In particular, I have a dataset X which is a 2D array. It ...
0
votes
0answers
9 views

Two Step estimation of treatment effects

I want to estimate the treatment effect of a selection Model without a preimplemented package in R. I allready have the probit estimates for the inverse mills ratio. How can I estimate the switching ...
1
vote
1answer
97 views

Least Squares Fits to Experimental Data

My attempt at making sense of the problem: The problem provides us with the sum of squares (SS; I believe that's the $18.1$). We can use the SS along with the number of samples $(21)$ to get the ...