Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...
5
votes
0answers
65 views
Aggregating all measurements per x-value in least-square fitting
We want to select one out of a given set of (continuous) functions that best matches a set of observations $\{(s_i,c_i) \mid 1 \leq i \leq N\} \subseteq\mathbb{N} \times \mathbb{N}$ (input size and ...
4
votes
0answers
69 views
Regression model with heteroskedasticity in both variables
I've been learning (lurking) from this site for a while and I finally have a question I haven't seen answered yet.
I'm doing a flight test and trying to fit the resulting data to linear line. From a ...
4
votes
0answers
115 views
Estimating parameters of an unknown PID controller
Say that I have your standard PID controller at work. To keep it extremely simple imagine I have a target $x^*$ on the variable $x$. Then the controller is:
$y(t) = K_p ( x^* - x_t) + K_i \int_0^t ...
4
votes
0answers
450 views
Methods to best test lead/lag relationships
I was wondering if you can share your experiences on what you feel is the best method to test lead / lag relationships between I(1) time series variables (i.e stock prices) and advantages and ...
3
votes
0answers
36 views
Reducing the dimension of an embedding
Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations.
Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
3
votes
0answers
119 views
R-squared in linear model verses deviance in generalized linear model?
Here's my context for this question: From what I can tell, we cannot run an ordinary least squares regression in R when using weighted data and the survey package. ...
3
votes
0answers
104 views
Tolerance interval for Deming regression
I am trying to derive (one-sided) tolerance intervals related to the Deming regression model:
$$ x_i=x^*_i + \epsilon_i$$
$$ y_i = (\alpha+\beta x^*_i) + \epsilon'_i$$
where the $x^*_i$'s are ...
3
votes
0answers
208 views
Definition and Convergence of Iteratively Reweighted Least Squares
I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form,
$J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$
where $N$ is the number of ...
3
votes
0answers
103 views
Relationship between regularized least squares and MLE
We know that the least square method is equivalent to the MLE for Gaussian distributed errors. What is the relationship (if any) between regularized (Tichonov regularization) least squares and MLE?
3
votes
0answers
88 views
Scope of non-linear least squares
edit: tl;dr: I can coerce a lot of optimization problems to take the form of a non-linear least squares problem, but does it make sense to do so?
Suppose we have some empirical data $P=\{(x_i', ...
3
votes
0answers
174 views
Standard errors in weighted least squares
My data is a panel of countries by year. Suppose my main RHS variable is a country's GDP and my main LHS variable classifies countries by whether the country is a democracy.
Is it desirably to ...
2
votes
0answers
76 views
Gauss-Markov theorem: BLUE and OLS
I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem.
We assume a linear model, in matrix form, is given by:
$$ y ...
2
votes
0answers
17 views
Time variation in coefficients
Given $x_t, y_t$ ($t=1,\ldots,240$), I want to estimate $y_t = \alpha_t + \beta_t x_t$ and test $H_0: \alpha_1=\ldots=\alpha_T=0$. It is crucial to allow for time variation in the regression ...
2
votes
0answers
72 views
Non negative least squares with minimal colinearity
I am trying to fit a dataset using the standard NNLS (non-negative least squares) approach.
Formally:
$\min_x ||Ax-b||^2_2$ s.t. $x\ge0$
This is a quadratic program and can be solved optimally. The ...
2
votes
0answers
29 views
The Effect of Confounders on OLS estimates?
I'd like to see in linear regression how not adjusting for confounders affects estimated coefficients?
Here is what my lecture notes say:
1) Un-adjusted model: $E[Yi] = \beta_0 + \beta_1X_i$
2) ...
2
votes
0answers
80 views
Literature on robustness of regression assumptions
In my OLS regression not all assumptions are perfectly met, but I read that due to a large sample size there is a certain robustness to assumptions (my sample is 2500 people).
E.g. the DV isn't ...
2
votes
0answers
1k views
What to do when ovtest and linktest in Stata suggest model misspecification?
I have a sample that consists of 50 observations. The base model of the OLS-Regression with three control variables, two of them significant, has a $R^2=0.50$ and its F-Value is 7. Both ...
1
vote
0answers
27 views
Out-of-sample vs. in-sample interpretation
I am running predictive regressions on stock returns and as expected less relations hold out-of-sample than in-sample, however in some cases I find a significant relation out-of-sample but not ...
1
vote
0answers
38 views
Under which assumptions does the ordinary least squares method give efficient and unbiased estimators?
Is it true that under the Gauss Markov assumptions the ordinary least squares method gives efficient and unbiased estimators?
So:
$$E(u_t)=0 $$ for all $t$
$$E(u_tu_s)=\sigma^2 $$ for $t=s$
...
1
vote
0answers
51 views
Forward Stepwise selection
I am assuming the following model:
$Y = \beta X + \epsilon$
Here both $X$ and $Y$ are matrices. I fit the least squares model without any regularization and get the matrix $\beta$. I would like to ...
1
vote
0answers
45 views
High heteroscedasticity level
My dependent variable - logincome - and one independent variable - age - are continuous. All other explanatory variables are categorical including BA_degree, race, occupation, region, homeownership.
...
1
vote
0answers
70 views
Confidence intervals (ex-ante) for not-least squere forecasting method
Assume that we know the model of physical experience (for example - the pendulum is moving forward - then the position $s = vt + b\sin(t)$, where $t$ is time and $v$ and $b$ do not matter).
We have ...
1
vote
0answers
103 views
Using least squares to estimate mean
From "Generalized Additive Models: an introduction with R" by Simon N. Wood, page 55, Exercise 1.9, Question 1:
4 car journeys in London of length 1, 3, 4 and 5 kilometres took 0.1,
0.4, 0.5 and ...
1
vote
0answers
92 views
Slope Derivation for the variance of a least square problem via Matrix notation
I have a question to solve the following matrix problem:
$$E[( (X'X)^{-1}X \epsilon )^T ((X'X)^{-1}X \epsilon )]$$
into the solution
$$= \Sigma^{2} (X'X)^{-1}.$$
Where $\Sigma$ is the covariance ...
1
vote
0answers
61 views
Trying to test the Significance of the Seattle Seahawks Home Field Advantage
Basically I have an interest in sports and as part of that do a Microsoft Excel based version of Kenneth Massey 1997 thesis (http://masseyratings.com/theory/massey97.pdf) method of ranking sports ...
1
vote
0answers
75 views
Is high multicollinearity always an issue in OLS?
$$Y_t = a + bX_{1,t} + cX_{2,t} + dX_{3,t} + e_t$$
A high $R^2$ in $X_{1,t} = \alpha + \beta X_{2,t} + \gamma X_{3,t} + \varepsilon_t$ will always result in a higher standard error of the $b$ ...
1
vote
0answers
107 views
What are the conditions where we can regress non-stationary variables?
Obviously there are certain spots where it's okay to include a non-stationary predictor variable in a linear regression model. For example, a dummy variable interacted with a stationary variable must ...
1
vote
0answers
272 views
Question about Hausman-test for endogeneity with two endogenous regressors with potential heteroscedasticity
First question: Is the following example of computing the Hausman-test for endogenity with two endogenous regressors adequate?
Second question: Is it true that in case of heteroscedasticity, i.e. ...
1
vote
0answers
65 views
Couple of linear regressions with heteroskedasticity and correlated errors
I have two linear models:
$$
y_1 = \mu_1 + \beta_1x + \epsilon_1
$$
$$
y_2 = \mu_2 + \beta_2x + \epsilon_2
$$
where $\epsilon_1, \epsilon_2 \sim \text{MVN}(0, \Sigma(x))$, where $\Sigma(x)$ in NOT ...
1
vote
0answers
165 views
When are the asymptotic variance of OLS and 2SLS equal?
Assume the model $ \ y = X\beta + u \ $ with $\ W \ $ is a $ \ n\times l \ $ so called matrix of instruments.
The following assumptions hold. There is a law of large numbers (LLN) for 1.,2.,3. and ...
1
vote
0answers
81 views
Two-stage least squares approach to Deming regression
I am interested in statistical inference for the Deming regression model:
$$ x_i=x^*_i + \epsilon_i$$
$$ y_i = (\alpha+\beta x^*_i) + \epsilon'_i$$
where the $x^*_i$'s are nonrandom fixed numbers, ...
1
vote
0answers
117 views
Correlation and coefficients in ols
I am trying to fit a simple linear model with OLS. I have say 4 terms and the number of data points is >>4, so no explicit overfitting.
1) My terms or my independent varibales do have a strong ...
1
vote
0answers
133 views
Doubt on Covariance Matrix in Weighted Least Square Estimation
This is a page from the book linear algebra, geodesy and gps by Gilbert Strang. The page explains about the justification of the inverse of the of the variance-covariance matrix of measurement ...
1
vote
0answers
215 views
Pregibon test for linearity vs. Ramsey's RESET test
Does every Ordinary Least Squares (OLS) regression model have to pass both the Pregibon Test for Linearity (sometimes called link-test) and the Ramsey RESET tests?
I am working on an OLS model and it ...
1
vote
0answers
147 views
Predictive power (or $R^2$) adjusted for certain variables
I will frame this question for Ordinary Least Square (OLS) regression, but my question is for both OLS and Logistic.
Let's say we data over 10000 different individuals. For each person we have three ...
0
votes
0answers
20 views
dickey-fuller and regressions
I have searched the internet on this, but I could not find any book/lecture/... that relates the ADF test and OLS regressions in practice.
Here are my questions:
1) it seems to me unclear what model ...
0
votes
0answers
32 views
Whats the biggest difference in calculating a simple regression model with/without a constant term?
I have calulated an OLS with and without a constant term. However, besides that the values are different I haven`t really found anything valueable?
Therefore my question is:
Whats the biggest ...
0
votes
0answers
27 views
ols regression on stationary series
I am trying to regress (OLS) some time series on stock returns. I am not interested in regressing the returns of those time series on my stock returns, but I want to include information about the ...
0
votes
0answers
33 views
Too many controls?
I am purposely keeping this question vague since this has happened to me with multiple datasets. I have a DV and a few IV then several controls, and all looks fine. Then I add one more controls and ...
0
votes
0answers
84 views
How does Newey-West covariance help increase accuracy of OLS estimates?
I have implemented a model using OLS estimates, but the results don't look too good. I've come across this term 'Newey-West covariance', and that I need to use residuals from my model as input, but ...
0
votes
0answers
67 views
Comparing ln(y) =bX vs. OLS and other count data models
My data is a balanced panel.
12 years X 600 ids/year X 12 months = 864,000 observations.
For about 150 of my 600 ids, at a certain point in this 12 year period a treatment is applied switching them ...
0
votes
0answers
32 views
Estimable function of OLS parameters can be shown by inner product with Null space?
I am in a advanced linear models class, and we are currently covering estimable functions. The criterion that we have for an estimable function is that for any $a^T\beta$ there exists an unbiased ...
0
votes
0answers
118 views
How to intuitively understand cov(xy)/var(x)
I asked this question the other day Understanding OLS regression slope formula. I am wondering if a similar explantion can me made for cov(xy)/var(x)? I realize the questions are similar, but I am ...
0
votes
0answers
32 views
Calculating a partial effect of one regressor in a non-linear (in the data) regression model?
The reason I'm asking this simple question is because the answer in my (professor's) slides seems to be somewhat incomplete and the book I use (introduction to econometrics, Stock & Watson) ...
0
votes
0answers
40 views
Comparing fuzzy RD estimates to an “OLS” analogue
In IV, I've sometimes seen the estimates compared to OLS to give a sense of how the LATE may compare to ATE.
What would the analogue be in fuzzy RD? What is the benchmark estimate?
I'd imagine ...
0
votes
0answers
137 views
Highly Collinear Independent Variables of Interest
Suppose I am interested in the follows:
I have county-level data. For each county, I know the share of the population that was born from one parent and the share of the population that was born from ...
0
votes
0answers
199 views
Introducing dummy variables to least squares estimation
I have a simple algorithm for identifying similar products based on product characteristics. When those characteristics are size, weight and price (ie continuous variables), a simple least squares ...
0
votes
0answers
63 views
OLS with the lowest MSE question
I am struggling to prove that the an estimator $\beta_{ls}=\frac{\sum X_iY_i}{\sum X_i^2}$ has a lower MSE than $\beta_{ols}=\frac{\sum (X_i-\bar{X})(Y_i-\bar{Y})}{\sum (X_i-\bar{X})^2}$.
The least ...
0
votes
0answers
114 views
The statement of homoscedasticity of variance when describing the OLS model
In an applied econometrics paper, the author states the model to be estimated as:
Why does the author claim homoscedasticity? This isn't making sense to me; can't the population variance-covariance ...
0
votes
0answers
304 views
Robust regression
Salut!
I implemented in Matlab, the following estimator $(k >0 )$:
$
\rho(x)=\begin{equation}
\left\{
\begin{array}{ll}
(k-1)^2 - \frac{(k-1)^4}{2 x^2} \text{if } x<-(k-1)\\
...
