New answers tagged least-squares
1
vote
How does expectation maximization relate to weighted least squares?
While applying EM on a mixture of Gaussian (like your case), part of the steps do coincide with weighted least squares.
E step: Taking expectation w.r.t. $C$ on the loglikelihood results in a "...
- 781
3
votes
Accepted
Does the matrix notation for OLS loss function assume that indexing a row from $X$ return a column or row vecotr?
Capital $X$ is the matrix of all feature values for all subjects, while lowercase $x$ is the vector of feature values for one subject. The lowercase $x$ is used in this notation to indicate that it is ...
- 58.3k
0
votes
Interpretation coefficients categorical variables
It appears that you have several binary explanatory variables, all of which are binary and enter the model either as simple main effects or a 2-way interaction.
For the coefficient of a main effect, ...
0
votes
How can I quantify uncertainty for a least squares estimator in a multivariate linear regression with covariance structure?
A general result from least squares estimation is that
$$\boldsymbol{\hat{\beta}}=(X'X)^{-1}X'\mathbf{y},$$ where we can treat $\mathbf{C}=(X'X)^{-1}X'$ as constant. Then, we can apply the variance ...
- 1,829
0
votes
ML vs WLSMV: which is better for categorical data and why?
Your question does not specifically reference factor analysis (FA) or structural equation modeling (SEM), though I will assume you are broadly interested in differences between estimators for ...
- 1,292
5
votes
When does adding a new predictor not increase $R^2$ in OLS?
This diagram from the answer at https://stats.stackexchange.com/a/113207/919 abstractly (but accurately) depicts the space $X_1$ spanned by all the (current) explanatory variables, the response ...
- 316k
1
vote
Accepted
How to prove OLS estimator still unbiased if omitted variable(s) are independent of included variables
This is just a question of whether $X_1^\top X_2=0.$
I.e. is every column of $X_1$ orthogonal to every column of $X_2$?
In particular, if an $y$-intercept term is included in the regression involving ...
- 8,796
1
vote
Accepted
How to model a standardized index in a regression?
With one caveat, there's no problem doing anything you would do with a regular variable to an index. After all, some 'regular' variables are indexes to start with. So as @patrick-coulombe notes, it'll ...
- 20.4k
2
votes
What is the objective function for weighted lasso & ridge?
It's not applied to the regulariser term.
Consider eg a repeated data set.. you can rewrite it with weights for the count of each data point (X,y) ie distinct row.
The weights would be on the error ...
- 5,974
3
votes
Homoskedasticity and Collinearity
There really isn't anything saying that these two things are explicitly related. You can have two predictors that are:
Almost perfectly collinear with heterogenous variance
Almost perfectly collinear ...
- 8,555
1
vote
Accepted
Why aren't OLS standard errors becoming smaller in the presence of serial correlation?
I figured out the problem. As Baltagi (2008) states in eq. 5.34 (p. 111), the estimated variance of the point estimate when serial correlation exists is the original estimator (i.e., s^2(X'X)^{-1}) ...
- 91
0
votes
Fixed Effects causes Multicollinearity
Multicollinearity means that the contribution of a variable (or a combination of variables) cannot be identified because it could be emulated by another variable (or a combination of them). Looking at ...
- 21k
0
votes
biased estimation of variable correlated with endogenous variable
Maybe I misunderstand the setup, but I do not directly see this spillover bias:
By linear projection theory, least squares on the equation for $Y$ will, for $D=(Z\; X)$ and $\hat\beta$ the vector of ...
- 31.7k
1
vote
Definition of fixed effects used in Journal Papers
This is an economics-flavored answer, which is appropriate given your research question. Other fields have their own jargon that is fairly different.
There are typically two ways to estimate fixed ...
- 34.7k
0
votes
Why not use instrumental variable directly as a covariate in the regression?
Well actually you could. Now suppose you have two groups of IVs, $Z_1$ and $Z_2$, both could be vectors. Suppose you include $Z_1$ in your regression with $Y$, then they're called "included ...
- 1
0
votes
Method of least squares, first order condition and QR decomposition
Taking into account the comments by @PBulls, @Zhanxiong and @whuber I was able to understand the problem I have and I was able to find the following reference The QR decomposition of a matrix that ...
- 11
1
vote
How best to estimate regression parameters subject to constraints?
Is one of these options preferable? If so, which one and why? Is one of these options so problematic that it should not be used? If so, which one and why?
Depends on your goal. If you trust your ...
- 268
Top 50 recent answers are included