New answers tagged

1 vote

How does expectation maximization relate to weighted least squares?

While applying EM on a mixture of Gaussian (like your case), part of the steps do coincide with weighted least squares. E step: Taking expectation w.r.t. $C$ on the loglikelihood results in a "...
Haotian Chen's user avatar
3 votes

Does the matrix notation for OLS loss function assume that indexing a row from $X$ return a column or row vecotr?

Capital $X$ is the matrix of all feature values for all subjects, while lowercase $x$ is the vector of feature values for one subject. The lowercase $x$ is used in this notation to indicate that it is ...
Dave's user avatar
  • 58.3k
0 votes

Interpretation coefficients categorical variables

It appears that you have several binary explanatory variables, all of which are binary and enter the model either as simple main effects or a 2-way interaction. For the coefficient of a main effect, ...
user1029384756's user avatar
0 votes

How can I quantify uncertainty for a least squares estimator in a multivariate linear regression with covariance structure?

A general result from least squares estimation is that $$\boldsymbol{\hat{\beta}}=(X'X)^{-1}X'\mathbf{y},$$ where we can treat $\mathbf{C}=(X'X)^{-1}X'$ as constant. Then, we can apply the variance ...
Ron Snow's user avatar
  • 1,829
0 votes

ML vs WLSMV: which is better for categorical data and why?

Your question does not specifically reference factor analysis (FA) or structural equation modeling (SEM), though I will assume you are broadly interested in differences between estimators for ...
Preston Botter's user avatar
5 votes

When does adding a new predictor not increase $R^2$ in OLS?

This diagram from the answer at abstractly (but accurately) depicts the space $X_1$ spanned by all the (current) explanatory variables, the response ...
whuber's user avatar
  • 316k
1 vote

How to prove OLS estimator still unbiased if omitted variable(s) are independent of included variables

This is just a question of whether $X_1^\top X_2=0.$ I.e. is every column of $X_1$ orthogonal to every column of $X_2$? In particular, if an $y$-intercept term is included in the regression involving ...
Michael Hardy's user avatar
1 vote

How to model a standardized index in a regression?

With one caveat, there's no problem doing anything you would do with a regular variable to an index. After all, some 'regular' variables are indexes to start with. So as @patrick-coulombe notes, it'll ...
conjugateprior's user avatar
2 votes

What is the objective function for weighted lasso & ridge?

It's not applied to the regulariser term. Consider eg a repeated data set.. you can rewrite it with weights for the count of each data point (X,y) ie distinct row. The weights would be on the error ...
seanv507's user avatar
  • 5,974
3 votes

Homoskedasticity and Collinearity

There really isn't anything saying that these two things are explicitly related. You can have two predictors that are: Almost perfectly collinear with heterogenous variance Almost perfectly collinear ...
Shawn Hemelstrand's user avatar
1 vote

Why aren't OLS standard errors becoming smaller in the presence of serial correlation?

I figured out the problem. As Baltagi (2008) states in eq. 5.34 (p. 111), the estimated variance of the point estimate when serial correlation exists is the original estimator (i.e., s^2(X'X)^{-1}) ...
Nicholas Ray's user avatar
0 votes

Fixed Effects causes Multicollinearity

Multicollinearity means that the contribution of a variable (or a combination of variables) cannot be identified because it could be emulated by another variable (or a combination of them). Looking at ...
Christian Hennig's user avatar
0 votes

biased estimation of variable correlated with endogenous variable

Maybe I misunderstand the setup, but I do not directly see this spillover bias: By linear projection theory, least squares on the equation for $Y$ will, for $D=(Z\; X)$ and $\hat\beta$ the vector of ...
Christoph Hanck's user avatar
1 vote

Definition of fixed effects used in Journal Papers

This is an economics-flavored answer, which is appropriate given your research question. Other fields have their own jargon that is fairly different. There are typically two ways to estimate fixed ...
dimitriy's user avatar
  • 34.7k
0 votes

Why not use instrumental variable directly as a covariate in the regression?

Well actually you could. Now suppose you have two groups of IVs, $Z_1$ and $Z_2$, both could be vectors. Suppose you include $Z_1$ in your regression with $Y$, then they're called "included ...
Elvisycc's user avatar
0 votes

Method of least squares, first order condition and QR decomposition

Taking into account the comments by @PBulls, @Zhanxiong and @whuber I was able to understand the problem I have and I was able to find the following reference The QR decomposition of a matrix that ...
luifrancgom's user avatar
1 vote

How best to estimate regression parameters subject to constraints?

Is one of these options preferable? If so, which one and why? Is one of these options so problematic that it should not be used? If so, which one and why? Depends on your goal. If you trust your ...
Sakari Cajanus's user avatar

Top 50 recent answers are included