Tagged Questions

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

1
vote
0answers
13 views

Least Squares Estimation of Poisson Parameter

"Assume independent random variables $Y_i$~$Poisson(λx_i)$. Supposing that $x_i$ are given, fixed constants, obtain the least squares estimator of $λ$ and compute its variance." This kind of a ...
1
vote
0answers
29 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
3
votes
1answer
102 views

Would a simple OLS regression work with this example?

First and foremost I would greatly appreciate any help you can provide me with. I am writing my undergraduate thesis on the rise of populism in France. The relationship I am trying to better ...
1
vote
1answer
24 views

Prove that System FGLS is Consistent

In the Systems of Equations framework, such as Seemingly Unrelated Regression (SUR), suppose we have $g=1,\ldots,G$ equations. Let $\mathbf{X}_i$ be a $G \times K$ matrix, $\mathbf{y}_i$ be $G \times ...
0
votes
1answer
41 views

Is the assumption of linearity necessary for the convergence of the least squares method to the MSE solution

More formally, say there are input vectors $\bf x$ and scalar outputs $Y$ being generated i.i.d. from a joint distribution $p$ and we are interested in estimating $\mu({\bf x}) = {\mathbb ...
1
vote
0answers
26 views

Why sandwich estimators aren't always used in OLS regression?

I asked before what is the intuition behind sandwich estimators. I must still missing something because I don't understand why sandwich estimators are not always applied to OLS residuals. Can you ...
8
votes
1answer
315 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
0
votes
2answers
35 views

Condition number of data matrix and stability of OLS estimates

I have a multivariate regression model $Y=X\beta ' + \epsilon$. The variables in the $X$ matrix have very different scales and hence the condition number of $X'X$ is huge (order of trillions). I ...
0
votes
1answer
25 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
0
votes
0answers
31 views

Weights in Weighted Least Squares Regression

Let's suppose I have the following linear model: $R = \beta * X + \epsilon$ I have a cross-sectional time-series sample of $n$ stocks, at $T$ periods of time. Not all stocks necessarily have data ...
5
votes
2answers
208 views

Does adjusting for superfluous variables bias OLS estimates?

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, ...
0
votes
1answer
31 views

Which post-hoc is more valid for multiple comparison of an unbalanced lmer-model: lsm or mcp?

After doing a model comparison with my mixed lmer model, I have a model with three main effects, no interaction, say ...
2
votes
1answer
27 views

Least Angle regression coefficient reaches zero after included

In LARS how is it possible that after including a variable it could reach zero again? http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf I understood that it works like: 1) choose ...
0
votes
0answers
39 views

Variance in significance with interaction term

I try to estimate the effects of NatRent (Natural resource rents in % of GDP) on GDP growth per capita (in %). When I include a Rule of Law (a measure for institutional quality) the coefficient of ...
1
vote
0answers
10 views

Help required in controlling for socioeconomic factors when mapping travel behavior

Background I have cleaned disaggregate travel data down to a household level (of about 75% of households) and socio economic data aggregated into 100-200 household blocks. What I'm wanting to do is ...
0
votes
1answer
23 views

Maximum Likelihood solution of a zero-covariance process

Let the measurement model be: $\tilde{y}=Hx+v$ $\tilde{y}=H\hat{x}+e$ where $H$ is the basis matrix, $v$ is a constant vector equal to, say, $a$, $x$ is the measurement variable and $e$ is a ...
1
vote
1answer
35 views

LSmeans - Unbalanced data with interactions

I wish to analyze an unbalanced data set with 3 variables Tleaf, Tair, and orientation (factor with two levels). Considering the effect of the factor "orientation", I wish to determine if "Tair" has a ...
1
vote
0answers
42 views

Estimating standard error of parameters of linear model fitted using gradient descent

Given a linear model $$y = X\beta + \epsilon$$ we can estimate parameters $\hat{\beta}$ using two different ways - ordinary least squares (OLS) and gradient descent (GD). Both of them boil down to ...
0
votes
0answers
7 views

How to compare plsr-selected components (generated from separate regression formulas)?

I would like to identify the subset of landscape variables from one distance class (within a series of different distance classes) which are the best predictors of 'noncol' abundance. There are many ...
3
votes
2answers
73 views

Function Approximation vs. Regression

Some background before I state the questions: I have a $d$-dimensional random vector $X=(X_1,\ldots,X_n)$ and a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$. Ultimately my goal is to understand $f$ ...
17
votes
2answers
491 views

Why does ridge estimate become better than OLS by adding a constant to the diagonal?

I understand that the ridge regression estimate is the $\beta$ that minimizes residual sum of square and a penalty on the size of $\beta$ $$\beta_{ridge} = (\lambda I_D + X'X)^{-1}X'y = ...
0
votes
0answers
100 views

How to derive OLS through MLE? [duplicate]

I am just curious on finding about this derivation
0
votes
0answers
27 views

Two-stage probit least squares

I am estimating a two-stage probit least squares (2SPLS) model. From the readings I have done so far, it appears the first stage of the 2SPLS has to be estimated with a probit, and then a continuous ...
0
votes
2answers
51 views

How to apply a logistic + an OLS model to the same data set?

I have a data set measuring rock detection depths $Y$ based on the distance from some point of interests $X$, which are classified based on geophysical criteria. Each observation $Y$ is set after ...
0
votes
0answers
21 views

Least-squares fitting with only optimum features, after Lasso - valid?

Using Lasso reduces the coefficients of features of a model, reducing some to zero, and thereby performing feature selection. The number of features depends on the value of $\alpha$ aka $\lambda$. In ...
5
votes
1answer
112 views

Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom

Background Suppose we have an Ordinary Least Squares model where we have $k$ coefficients in our regression model, $$\mathbf{y}=\mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$ where $\mathbf{\beta}$ ...
1
vote
0answers
18 views

Least Squares Regression - Error [duplicate]

In standard least squares regression, we find constants $\beta_1$ and $\beta_2$ such that the square of the average error, $\epsilon = y_i - (\beta_1 + \beta_2x_i)$, is minimized, and so the 'line of ...
0
votes
0answers
29 views

Help in calculating OLS estimators under certain constraints/modifications

I am new to these forums and to econometrics as a whole. I was hoping someone would be able to give me a nudge in the right direction with this problem. I've done extensive research both online and in ...
2
votes
0answers
54 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...
2
votes
2answers
134 views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
0
votes
1answer
51 views

What am I missing in basic OLS? [closed]

I must first admit that I haven't done stats in a loooong time, but I need to do some rudimentary analyses. My issue is that my findings just don't "look right," so I want to see if anyone can spot ...
0
votes
0answers
18 views

multidimensional time series nonlinear parameter estimation

I am trying to fit time series data for performing parameter estimation of a nonlinear multidimensional dynamical model (grey-box). At the moment I'm successfully using MATLAB's ...
0
votes
1answer
38 views

R: difference between Generalized Least Square and the Standard Least Squares with Cholesky

According to Wikipedia (source of all truth and knowledge...), http://en.wikipedia.org/wiki/Generalized_least_squares#Properties a weighted least square regression is equivalent to a standard least ...
1
vote
1answer
105 views

OLS vs IV estimates - Sign and Significance

Assume I have an equation with 1 endogenous variable, and many other exogenous variables. Also assume I have 2 valid instruments for the endogenous variable for IV estimation. If I were to estimate ...
1
vote
0answers
23 views

Orthogonal projection of a vector which is already orthogonal to part of the basis

Context This question emerged from trying to solve problem 5.1. of Wooldridge, Econometric Analysis of Cross Section and Panel Data. The problem asks to show the equivalence of the estimators ...
0
votes
1answer
91 views

How can you prove that the naive estimator is less efficient than the OLS estimator

The "naive estimator" is an estimate of the slope obtained by joining the first and last observations and dividing the increase in the height by the horizontal distance between them. Given that the ...
0
votes
0answers
11 views

Arbitrary least squares fitting with an initial model

I have a problem that seems like a straightforward implementation of linear least-squares, but I can't quite figure out how it will work. I have a (noisy) dataset $f(x,y)$ where $f$ is is non-linear ...
1
vote
1answer
51 views

Regressing a differenced variable on a lagged variable. How can I fix the error in R?

I have a time series (std) of 324 observations with no missing values, starting from January 1987 and ending in December 2013. I want to regress via OLS the one in the question. In R, the code: ...
0
votes
1answer
19 views

Why do we use fixed effects and controls at the same time?

When we run a regression model (say OLS for simplicity) of y ~ x, we might have to use several control variables say z1 and z2. Now our model is y ~ x + z1 + z2, we may believe that z1 and z2 are not ...
1
vote
0answers
28 views

How much does the value of a car depreciate due to time (only), and mileage (only)?

I have a database of second hand cars. It contains (among other things) the asking price, the mileage, the fuel consumption and the year it was built. I would like to know how much the value ...
1
vote
1answer
44 views

Regression model result interpretation

I am trying to learn few things about generalized least square regression modeling. Here are the models that I am using., X1 ~ Y1 + Y2, X2 ~ Y1 + Y4, All predictor variables, Y1, Y2 and Y4 are ...
3
votes
1answer
36 views

Prediction error in least squares with a linear model

In the classical linear model with $$Y=X\beta +\epsilon,$$ where $Y \in \mathbb{R}^n$ is the observation, $X\in \mathbb{R}^{n\times p}$ is the known covariates, $\beta \in \mathbb{R}^p$ is the ...
1
vote
2answers
72 views

Does least squares regression imply normality of errors?

For the linear model $$y_i=\beta_0 +\sum_{k=1}^{n}\beta_k x_{ik} + \epsilon_i$$ the parameter estimates are the same for the maximum likelihood method and the least square method (minimizing ...
1
vote
2answers
60 views

What is the effect of having skewed dependent variable on scatterplot result

The histogram of my dependent variable is as following: I draw the scatter plot of my dependent variable and independent variable, and the result is as following picture? I am wondering if the ...
3
votes
2answers
55 views

what is difference between ordinary least squares and residuals

I think the ordinary least squares are the sum of the vertical distance between the observed data to the model line(regression). And residual is calculated by add up all vertical distance between each ...
5
votes
1answer
105 views

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
2
votes
1answer
40 views

In OLS is it methodologically correct to use the variance of a variable as an explanatory variable?

Are some OLS assumptions not satisfied if I use the variance of a variable as a proxy of uncertainty in a regression? For instance, would it be methodologically correct if I use moving averages of ...
2
votes
2answers
111 views

Is there a result showing a relation between the size of the residuals and the correlation coefficient?

Consider an OLS regression between two variables. Is there any result which relates the size of the residuals (measured, perhaps, by the sum of the squares) to the Pearson correlation coefficient of ...
0
votes
0answers
25 views

Does this kind of overlap between in-sample data and forecast cause inflated $R^2$?

I am using a simple UIP model to forecast exchange rates using interest rates with a twelve month horizon. The equation I use is: $E(t+12) - E(t) = α + β(I*(t) - I(t))$. I apply OLS linear regression ...
0
votes
0answers
25 views

Is there a way to model the error term in a linear regression with R?

I'm tryin to estimate a pretty basic regression. I have a dataset containing x and y and would like to esimate the following ...