# Tagged Questions

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

94 views

### Bound for Arithmetic Harmonic mean inequality for matrices?

NOTE: This question has originally been posted in MSE, but it did not generate any interest. It was first posted there, because the question itself is a pure matrix-algebra question. Nevertheless, ...
112 views

### Regression model with heteroskedasticity in both variables

I've been learning (lurking) from this site for a while and I finally have a question I haven't seen answered yet. I'm doing a flight test and trying to fit the resulting data to linear line. From a ...
71 views

### Aggregating all measurements per x-value in least-square fitting

We want to select one out of a given set of (continuous) functions that best matches a set of observations $\{(s_i,c_i) \mid 1 \leq i \leq N\} \subseteq\mathbb{N} \times \mathbb{N}$ (input size and ...
69 views

### OLS robust to outliers

I am facing the following problem: I have a training sample and estimate a model on that training sample. My model is simply OLS: $y_t = a + \beta x_t + \varepsilon_t$. The model is estimated on ...
96 views

### Generalised least squares: from regression coefficients to correlation coefficients?

For least squares with one predictor: $y = \beta x + \epsilon$ If $x$ and $y$ are standardised prior to fitting (i.e. $\sim N(0,1)$), then: $\beta$ is the same as the Pearson correlation ...
247 views

### Definition and Convergence of Iteratively Reweighted Least Squares

I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form, $J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$ where $N$ is the number of ...
134 views

218 views

### Standard errors in weighted least squares

My data is a panel of countries by year. Suppose my main RHS variable is a country's GDP and my main LHS variable classifies countries by whether the country is a democracy. Is it desirably to ...
78 views

### How can the residuals and the fits of a regression model be correlated?

In the context of statistical regression and OLS, I understand that $Y$ lives in a n-dimensional space and has variance $\sigma^2$. The Hat Matrix, $H$, corresponds to an orthogonal projection of a ...
33 views

### Hypothesis test with parameter uncertainty

Basic problem but I can't seem to figure out how to think about this: In a simple OLS problem the estimate of parameter $\beta$ is $\beta^*=(X'X)^{-1}X'Y$. Then, the standard error of $\beta^*$ is ...
27 views

### Time variation in coefficients

Given $x_t, y_t$ ($t=1,\ldots,240$), I want to estimate $y_t = \alpha_t + \beta_t x_t$ and test $H_0: \alpha_1=\ldots=\alpha_T=0$. It is crucial to allow for time variation in the regression ...
228 views

### When are the asymptotic variance of OLS and 2SLS equal?

Assume the model $\ y = X\beta + u \$ with $\ W \$ is a $\ n\times l \$ so called matrix of instruments. The following assumptions hold. There is a law of large numbers (LLN) for 1.,2.,3. and ...
31 views

### The Effect of Confounders on OLS estimates?

I'd like to see in linear regression how not adjusting for confounders affects estimated coefficients? Here is what my lecture notes say: 1) Un-adjusted model: $E[Yi] = \beta_0 + \beta_1X_i$ 2) ...
116 views

### Literature on robustness of regression assumptions

In my OLS regression not all assumptions are perfectly met, but I read that due to a large sample size there is a certain robustness to assumptions (my sample is 2500 people). E.g. the DV isn't ...
2k views

### What to do when ovtest and linktest in Stata suggest model misspecification?

I have a sample that consists of 50 observations. The base model of the OLS-Regression with three control variables, two of them significant, has a $R^2=0.50$ and its F-Value is 7. Both ...
27 views

### Weighted least squares for energy data

I am trying to do a weighted multiple least squares regression on utility data. Basically I have utility bills where I use : Days billed, Consumption. For the same billing period I calculate Heating ...
24 views

### What theory of estimation does apply when estimation follows model selection?

Almost two decades ago, Chatfield JRSS(1995)[vol.158,p.441] wrote that ...
65 views

### Large difference between OLS and quantile regression coefficient?

I've have a variable that has a regression coefficient of 5 using OLS, but when I use quantile regression (examining every 5th percentile, 5, 10, 15, etc.), I find a coefficient that is anywhere from ...
26 views

### Obtaining an estimate of slope error on a linear fit to data with y error

I have a problem similar to what was discussed here: I have a dataset (x_i,y_i) with y-error which is nonuniform over the interval of interest. I want to fit a line to this and extract a value of ...
76 views

### Different Parameter Estimates of SAS and Minitab for Unreplicated Factorial Designs

I am using famous example from Myers and Montgomery (1995), which is $2^{4}$ factorial design to compare some methods for unreplicated factorial designs concerning active effects and mean squared ...
28 views

### IRLS working weights proportional odds

According to Reduced-rank vector generalized linear models the parameter estimates are obtained by Fisher-scoring algorithm. In the VGAM package the IRLS algorithm ...
177 views

### Using Pandas and statsmodels for ordinary least squares

Apologies in advance for the tedious beginner question. I'm trying to translate a least-squares problem from a manual process (using Excel for matrix transposition and multiplication) to using the ...
69 views

### How can I change my fitting method to force better relative accuracy instead of just absolute accuracy?

I have some data which I want to use for interpolation and extrapolation. I would (very much) prefer a function that I can plug values in instead of using cubic splines or some such technique so I ...
62 views

### Percentage change vs. first difference

I want to regress (using an OLS regression) survey results on stock returns. How do I decide whether I use the first difference or the percentage change of the survey results?
78 views

### How to choose initial values for nonlinear least squares fit

The question above says it all. Basically my question is for a generic fitting function (could be arbitrarily complicated) which will be nonlinear in the parameters I am trying to estimate, how does ...
59 views

### Newey-West t-stats and critical values

When using Newey-West standard errors for my t-statistics of the slope coefficients in an OLS regression, can I still use the usual critical values for two-sided tests? (1.645 for 10% significance, ...
75 views

### Out-of-sample vs. in-sample interpretation

I am running predictive regressions on stock returns and as expected less relations hold out-of-sample than in-sample, however in some cases I find a significant relation out-of-sample but not ...
97 views

### Under which assumptions does the ordinary least squares method give efficient and unbiased estimators?

Is it true that under the Gauss Markov assumptions the ordinary least squares method gives efficient and unbiased estimators? So: $$E(u_t)=0$$ for all $t$ $$E(u_tu_s)=\sigma^2$$ for $t=s$ ...
77 views

### Forward Stepwise selection

I am assuming the following model: $Y = \beta X + \epsilon$ Here both $X$ and $Y$ are matrices. I fit the least squares model without any regularization and get the matrix $\beta$. I would like to ...
63 views

### High heteroscedasticity level

My dependent variable - logincome - and one independent variable - age - are continuous. All other explanatory variables are categorical including BA_degree, race, occupation, region, homeownership. ...
104 views

### Comparing ln(y) =bX vs. OLS and other count data models

My data is a balanced panel: 12 years $\cdot$ 600 ids $\cdot$ 12 months = 86,400 observations. For about 150 of my 600 ids, at a certain point in this 12 year period a treatment is applied switching ...
78 views

### Confidence intervals (ex-ante) for not-least squere forecasting method

Assume that we know the model of physical experience (for example - the pendulum is moving forward - then the position $s = vt + b\sin(t)$, where $t$ is time and $v$ and $b$ do not matter). We have ...
127 views

### Using least squares to estimate mean

From "Generalized Additive Models: an introduction with R" by Simon N. Wood, page 55, Exercise 1.9, Question 1: 4 car journeys in London of length 1, 3, 4 and 5 kilometres took 0.1, 0.4, 0.5 and ...
117 views

### Slope Derivation for the variance of a least square problem via Matrix notation

I have a question to solve the following matrix problem: $$E[( (X'X)^{-1}X \epsilon )^T ((X'X)^{-1}X \epsilon )]$$ into the solution $$= \Sigma^{2} (X'X)^{-1}.$$ Where $\Sigma$ is the covariance ...
100 views

### Trying to test the Significance of the Seattle Seahawks Home Field Advantage

Basically I have an interest in sports and as part of that do a Microsoft Excel based version of Kenneth Massey 1997 thesis (http://masseyratings.com/theory/massey97.pdf) method of ranking sports ...
97 views

### Is high multicollinearity always an issue in OLS?

$$Y_t = a + bX_{1,t} + cX_{2,t} + dX_{3,t} + e_t$$ A high $R^2$ in $X_{1,t} = \alpha + \beta X_{2,t} + \gamma X_{3,t} + \varepsilon_t$ will always result in a higher standard error of the $b$ ...
463 views

### Question about Hausman-test for endogeneity with two endogenous regressors with potential heteroscedasticity

First question: Is the following example of computing the Hausman-test for endogenity with two endogenous regressors adequate? Second question: Is it true that in case of heteroscedasticity, i.e. ...
80 views

### Couple of linear regressions with heteroskedasticity and correlated errors

I have two linear models: $$y_1 = \mu_1 + \beta_1x + \epsilon_1$$ $$y_2 = \mu_2 + \beta_2x + \epsilon_2$$ where $\epsilon_1, \epsilon_2 \sim \text{MVN}(0, \Sigma(x))$, where $\Sigma(x)$ in NOT ...
93 views

### Two-stage least squares approach to Deming regression

I am interested in statistical inference for the Deming regression model: $$x_i=x^*_i + \epsilon_i$$ $$y_i = (\alpha+\beta x^*_i) + \epsilon'_i$$ where the $x^*_i$'s are nonrandom fixed numbers, ...
59 views

### Comparing fuzzy RD estimates to an “OLS” analogue

In IV, I've sometimes seen the estimates compared to OLS to give a sense of how the LATE may compare to ATE. What would the analogue be in fuzzy RD? What is the benchmark estimate? I'd imagine ...
155 views

### Correlation and coefficients in ols

I am trying to fit a simple linear model with OLS. I have say 4 terms and the number of data points is >>4, so no explicit overfitting. 1) My terms or my independent varibales do have a strong ...