Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

2
votes
0answers
8 views

Least squares / residual sum of squares in closed form [migrated]

In finding the Residual Sum of Squares (RSS) We have: \begin{equation} \hat{Y} = X^T\hat{\beta} \end{equation} where the parameter $\hat{\beta}$ will be used in estimating the output value of input ...
0
votes
0answers
18 views

White standard errors and Weighted Least Squares (WLS) [on hold]

Are there situations in which it would be useful to apply WLS and use White standard errors (from the transformed model) as well?
2
votes
0answers
32 views

Which observation has the largest variance of the residual?

a) For which of the following observations (obs1, obs2, obs3) is the variance of the residual the largest? Which observation has the highest leverage? And which one the smallest? Explain why. ...
0
votes
1answer
35 views

Least Squares Question to show proportion

I have a question. Let's say data has been collected for people with jobs and people without jobs. People with Jobs are represented by 1, people without jobs are represented by 0. How do you use a ...
0
votes
4answers
61 views

How to interpret a regression that includes GDP, GDP per capita, and population

While GDP per capita = GDP / population and are obviously related, these three measurements are not perfectly collinear and can be included in OLS just fine. ...
1
vote
1answer
32 views

Could standardizing an independent variable cause the t-statistic of the OLS estimate to change?

Lets say we are looking to estimate the following standard OLS regression: $y_{i} = \beta_{0} + \beta_{1}*X_{i} + \beta_{2}*Z_{i} + \epsilon_{i}$ and that we choose to standardize $X$ as: ...
0
votes
0answers
16 views

pooled ols when the observations pooled are the same countries

My question is: is it possible to pool observations when it is the same countries that are observed through the years. I have observations on 37 countries in 2010, 47 countries in 2011 and 60 ...
0
votes
0answers
32 views

Parameter uncertainty after non-linear least squares estimation

I've fit a system of non-linear ODE to some experimental data using Levemberg-Marquardt. After the algorithm converged, I estimated the Hessian matrix of the system using: $H = (J^TJ)$ The ...
1
vote
1answer
35 views

Multicollinearity if independent variables sum to one?

I am intending to explain the variation in dependent variable Y through a number of ratios that, combined, sum to 1 in each period t Hence, $$ Y_t = \alpha + ...
1
vote
1answer
64 views

Link between linear regression and correlation

I happen to do a lot of linear regression, the typical use case is predicting $Y$ with a few predictors say $X = [X_1;X_2;X_3]$. So we we find $\hat{\beta}$ such that $$\hat{\beta} = ...
1
vote
0answers
34 views

Setting up a naiv tensor product B-spline example

I've simulated data according to $y = \text{sin}(2\cdot(4x-2))+2\cdot\text{exp}(-(16^2)(x-0.5)^2)+\epsilon$ where $\epsilon \sim \mathbb{N}(0,0.3^2)$ By evaluating the ith B-Spline of degree $k$, ...
0
votes
1answer
27 views

Cross product term in linear regression why is it zero

I'm trying to see why this is the case. I have taken the LHS and added $X\hat{\beta}$ and subtracted $X\hat{\beta}$. From this I can get both terms on the right but I end up with a cross product ...
2
votes
0answers
17 views

3D plot of the residual sum of squares in linear regression [migrated]

I'm trying to reproduce Figure 3.2 from the book Introduction to Statistical Learning. Figure describes 3D plot of the residual sum of squares (RSS) on the Advertising data, using ...
1
vote
0answers
32 views

2-stage Heckman instrumental variable estimation

I am working on my thesis. My main regression model is the following: $Y=x_1*{\rm Payment}+x_2*{\rm Country}+x_3*{\rm Industry}...$ All independent variables are dummy / binary variables. In a next ...
1
vote
1answer
41 views

Link Functions where MLE=OLS

This is a follow-up of a question that I posted previously. I'm trying to get parameter estimates from two different SAS functions (Proc REG and ...
1
vote
2answers
54 views

SAS - REG vs GENMOD; OLS vs MLE

I'm using a very simple data set from an article in trying to further my understanding of GLMs. I've input the data using SAS, and I've run both the PROC REG and PROC GENMOD procedures on the data. ...
1
vote
0answers
36 views

OLS with a heavily skewed independent variable

I am regressing a log-normally distributed dependent variable (wage) on a heavily skewed independent variable and I want to make sure I handle it in the best ...
0
votes
0answers
12 views

Using panel data with different variables: SUR or OLS separatedly?

I have a merged database from two cross-sectional surveys for distinct years ($t_1$ and $t_2$). If all questions were exactly the same, I could just stack the data horizontally and run a normal panel ...
0
votes
0answers
23 views

Should the control variables in the first and second stage of 2SLS be different?

I first noticed this when I was running some models is Stata and the variables that I had used as controls in the second stage were showing up in the first stage (I think this is just Stata's default ...
1
vote
1answer
64 views

OLS vs. quantile regression

I ran OLS regression in Stata. Based only on the results I got in OLS, is there any way to know if the quantile regression will be a better choice?
4
votes
0answers
26 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
1
vote
1answer
30 views

Trouble in comprehending question: Simple linear regression - LS and MLE

I've been given the simple linear regression model: $y_i = β_0 + β_1x_i + ε_i$ Under the assumptions of a simple linear regression model, the question they ask is: Assuming the usual model ...
1
vote
1answer
24 views

What can be inferred when multivariable ordinary least squares and quantile (median) regression yield differing results?

There lies information in a discrepancy of the (unconditional) mean and median. For example, if the median is larger than the mean, the distribution must be left-skewed. Does this kind of inference ...
1
vote
0answers
40 views

Minimum observations for creating dummy variables from a categorical group variable

I have an OLS regression with around 1000 observations and created a dummy variable for at least 15 different categories (catholic, muslim, hindu etc). One of the created dummy variables (spiritual) ...
2
votes
1answer
101 views

ML vs WLSMV: which is better for categorical data and why?

I was wondering which is a better estimator to use for categorical data: ML or WLSMV. I saw on a discussion on the Mplus website that they recommend WLSMV for categorical data but didn't explain why. ...
0
votes
0answers
13 views

Least-squares, pivoting/rotating around a reference point?

Suppose I have two datasets $y1$ and $y2$ sampled equally on $x$. I would like to bring $y2$ in register with $y1$ in a least-squares sense, and referenced to one point at $x_{ref}$ in $y1$ (i.e. the ...
3
votes
1answer
45 views

Finding clusters to fit least squares and produce a piecewise equation

In the figure below, I've manually drawn an approximate solution to a least squares fit of the associated regions separated by black lines. The data appears to be bounded by two asymptotes (y=-18 and ...
3
votes
3answers
105 views

Looking for a proof that overfitting a model leads to greater variance estimates (under OLS)

So I've been trying to algebraically prove that overfitting a model leads to greater variance values for the parameter estimates. I've gotten close (reduced the problem to showing a certain matrix is ...
1
vote
1answer
60 views

Recommendation for sequential least squares programming book

I would like to understand better how to solve least squares in a sequential/recursive way. How to use weights in the least squares if the following problem: ...
1
vote
0answers
18 views

Constructing the approximate confidence set for parameter vector beta in least square regression

From Elements of Statistical Learning, on page 49 (in the context of least square regression), we are given the approximate confidence set (shown in image) for the parameter vector $\beta$. I want ...
2
votes
2answers
124 views

Standard error of regression coefficients without an assumption of homoscedastic normal noise

I have a time series that is affected by two (or more) kinds of events. When event $A$ happens, some signal is linearly added to the time series (the signal lasts, for example, for 100 time points). ...
0
votes
1answer
65 views

How do you calculate the Ordinary Least Squares estimated coefficients in a Multiple Regression Model? [duplicate]

I recently began learning about OLS estimation of multiple regression models and came across the following formulas explaining the calculations: What would the formulas be for an OLS regression ...
1
vote
0answers
43 views

Proof for “Least squares estimator is BLUE”

This question was raised before. But so far I haven't got any satisfactory answer yet. So I raise it again. Suppose we have the linear model $$ y = X \beta + v, \quad v \sim (0, \sigma^2 I), $$ where ...
0
votes
2answers
66 views

Least Squares Dummy Variable Regression dropping states/variables?

So I am running a Least Squares Dummy Variable Regression (LSDV1) involving data from 21 states observed 3 times (2007, 2008, 2009) and dropping one of the dummy value states. I have 8 independent ...
1
vote
0answers
22 views

Unbiased estimator of projection coefficient

A question emerging from my reading of Hansen's econometric textbook. Given a random variable $y$, a random vector $x$, the solution to $\min_{\beta} [y - x' \beta]^2$ is known to be $\beta = ...
3
votes
0answers
37 views

Tobit or OLS question

My dependent variable is mostly continuous and positive, but has a modest number of zeros (10% of the sample). The results from tobit and ols are very similar. How can I formally compare the tobit and ...
2
votes
1answer
76 views

OLS estimate of a linear model with dummy variable

I know a regression of y on x (dummy variable) and a constant term can be represented in the following form: On the other hand OLS estimator can be presented in the following form: I need to see ...
2
votes
1answer
77 views

Proof for “Least squares estimator is BLUE”

I checked all the books and on-line materials I could find for the proof, but found all of them have a derivation problem, which I cannot understand. To prove the least squares estimator is the BLUE ...
0
votes
0answers
18 views

Correlated Regressors - Motorway Accidents [duplicate]

What is the best way to deal with correlated regressors? I am running a regression with No. of Accidents as the Dependent Variable and Traffic Flow and Speed as the independent variables. The ...
3
votes
1answer
149 views

Using MLE vs. OLS

When is it preferable to use Maximum Likelihood Estimation instead of Ordinary Least Squares? What are the strengths and limitations of each? I am trying to gather practical knowledge on where to ...
0
votes
0answers
16 views

Regression analysis (OLS) in matched case-control study

I have been asked to assist in a study based on samples of records from registers of dental care patients. My problem is that I would like to be sure that my proposed approach for the analysis is ...
2
votes
2answers
80 views

Difference between OLS and FE

I have a more general question. Could somebody please explain what is the general difference between OLS and FE (Fixed Effects) in a very simple way? In terms of use in panel data and in general. ...
3
votes
4answers
127 views

Can the use of dummy variables reduce measurement error?

If the continuous variables are measured with error, can the use of dummy variables mitigate the problem? For instance, IQ measures intelligence with error. So will using a dummy of high, medium, low ...
1
vote
1answer
92 views

Solving regularized least squares problems using Matlab optimization toolbox

I am trying to solve a least squares problem where the objective function has a least squares term along with L1 and L2 norm regularization. I am unable to find which matlab function provides the ...
2
votes
1answer
199 views

Calculate log-likelihood “by hand” for generalized nonlinear least squares regression (nlme)

I'm trying to calculate the log-likelihood for a generalized nonlinear least squares regression for the function $f(x)=\frac{\beta_1}{(1+\frac x\beta_2)^{\beta_3}}$ optimized by the ...
0
votes
1answer
44 views

Seemingly Unrelated Regression vs Systems OLS

Could someone please clarify the relationship (if any) between SUR and SOLS? Is SOLS a method for estimating SUR? What are the differences between SUR and SOLS? Also, is it true that these methods ...
1
vote
1answer
62 views

Cholesky update for removing row

What is most efficient way to updating cholesky factorization of for removing a column from the matrix? T = Chol(X' *X) If I remove a column from X, how to ...
1
vote
1answer
44 views

Asymptotic Least Squares question (with random regressors)

Consider the DGP $y_i=x_i+\epsilon_i$, where $\epsilon_i \sim Z$. We estimate $\beta=1$ by regression without a constant term, so in $y_i=\beta x_i + \epsilon_i$. Show that this DGP does ...
2
votes
1answer
79 views

Can we still use OLS on truncated a $Y$ if its conditional distribution is normal?

I was recently reading about Heckman selection models, and got sidetracked by how little I knew about truncated data. I was reading these slides, and on page 78 Baum mentions that if part of a sample ...
14
votes
3answers
780 views

What does “all else equal” mean in multiple regression?

When we do multiple regressions and say we are looking at the average change in the $y$ variable for a change in an $x$ variable, holding all other variables constant, what values are we holding the ...