Timeline for What measure of effect size in ANOVA has mode at zero under the null (unlike $\eta^2$ that does not)?
Current License: CC BY-SA 3.0
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 13, 2017 at 12:44 | history | edited | CommunityBot |
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
|
|
| Dec 23, 2014 at 17:28 | history | edited | amoeba | CC BY-SA 3.0 |
updated explaining a possible application
|
| Dec 23, 2014 at 14:45 | comment | added | Michael M | @amoeba: its expectation is 0, not its mode. As soon as you think of out-of-sample variance explained, negative sample values immediately make sense. | |
| Dec 23, 2014 at 14:36 | comment | added | amoeba | @Michael: is it really? I have been reading on it since yesterday, and it seems that adjusted R squared $\bar{R}^2$ uses unbiased estimates of the error and total variances (instead of the biased ones that enter $R^2$), but the ratio of them will still be biased, making $\bar{R}^2$ biased as well, even if less so than $R^2$. However, the most annoying thing about $\bar{R}^2$ is that it can be negative (and of course it has to be in order to be even approximately unbiased). I am trying to understand whether I can maybe use an unnamed (?) quantity $\mathrm{max}\{0, \bar{R}^2\}$ though. | |
| Dec 23, 2014 at 14:29 | comment | added | Michael M | You are probably aware that R-squared adjusted is expected to be zero under the null? | |
| Dec 23, 2014 at 12:01 | history | edited | amoeba | CC BY-SA 3.0 |
important updates to clarify
|
| Dec 23, 2014 at 2:43 | history | edited | Sycorax♦ | CC BY-SA 3.0 |
edited title
|
| Dec 23, 2014 at 2:24 | answer | added | gung - Reinstate Monica | timeline score: 5 | |
| Dec 23, 2014 at 1:02 | comment | added | amoeba | I will think about how to edit this question, but have already posted a separate question as you suggested: What is the distribution of $R^2$ in linear regression under the null hypothesis?. My hope is that somebody will quickly answer that question and it will help me edit this one. | |
| Dec 23, 2014 at 0:34 | comment | added | Silverfish | I hope this discussion has helped clarify some things for you! This has the basis for a really good question though I suggest you edit to clarify re bias. You should probably ask about the null distribution of $R^2$ as a separate question - I'm surprised it hasn't been asked here before but can't see it. (The F stat can be written in terms of $R^2$, which gives a clue!) This simulation may be instructive for you. | |
| Dec 23, 2014 at 0:16 | comment | added | amoeba | I see. You are of course right about the meaning of "biased", thanks for correcting this confusion! I have no idea what the better term might be, let's call it "mode-biased" for the time being. So my questions about regression then become: is $r^2$ mode-biased for simple regression? is $R^2$ mode-biased for multiple regression? If the answers to these two questions are not the same, then why? | |
| Dec 23, 2014 at 0:03 | comment | added | Silverfish | When I see "$\hat{\theta}$ is biased", I think this means $\mathbb{E}(\hat{\theta})\neq\theta$. Regarding $R^2$ as an estimator of the % variation explained in the population by the correct model, or $R$ as an estimator of the correlation between fitted and observed values in the population, the concept of them being "biased" estimators does make sense. However it relates to mean not mode (which is why I wonder if you're looking for a different word than "bias") and it's "obvious" they're biased if $H_0$ is true: since they're positive they can't average out to zero. | |
| Dec 23, 2014 at 0:01 | comment | added | amoeba | Yes, I believe $\eta^2$ is $R^2$. So let's talk about regression then; as far as I understand, a t-test corresponds to a regression with only one predictor. It is obvious that $r$ (correlation coefficient) will peak at zero under null. But what will be the mode of $r^2$ distribution under the null? If it is also zero, then one part of my question is: why is this different in multiple regression? Note that I am not asking about $\mathbb{E}(R^2)$, as you wrote above; I am asking about the mode. | |
| Dec 22, 2014 at 23:54 | comment | added | Silverfish | I'm afraid every time I see an ANOVA I default to thinking about a regression! Users whose brains work in "ANOVA mode" are better-placed to answer this question than I am, but $R^2$ and $\eta^2$ are closely linked. | |
| Dec 22, 2014 at 23:48 | comment | added | amoeba | @Silverfish, I am not very familiar with this topic, so it might very well be that my questions are naive. Sorry for that and thanks for understanding. But I am confused. Is $\eta^2 = R^2$ going to peak at zero for a t-test? If yes, then how does it relate to what you wrote? Or should I maybe be thinking about $r$ and not about $r^2$? Wait, is then the problem that for many groups / many predictors there is no way of "taking a square root" from $R^2$ and assigning it a meaningful sign? | |
| Dec 22, 2014 at 23:42 | comment | added | Silverfish | "is unbiased (peaks at zero under null hypothesis)": I may be missing something here, but relating this back to multiple regression, it's almost impossible for $R^2 = 0$ even under $H_0$. You'd have to get lucky with your $X$ being orthogonal to observed $Y$. Demanding that $\mathbb{E}(R^2) = 0$ under the null is obviously an unreasonable requirement because it is almost always going to be positive, but never negative (similar but not identical considerations for $R^2_{adj}$). As $\rho^2$ is zero under the null, $R^2$ is unsurprisingly biased. But perhaps you mean something else by "biased"? | |
| Dec 22, 2014 at 23:17 | history | asked | amoeba | CC BY-SA 3.0 |