Tag Info

Accepted

Interpretation of p-value near alpha level

There are two different approaches to interpreting statistical significance - the Fisher way, and the Neyman-Pearson way. We smush these together (into what Gerd Gigerenzer has called a 'bastardised ...
• 14.8k
Accepted

Can an irrelevant variable be significant in a regression model?

Here is a good example. Suppose you are interested in modelling the effect of ice cream sales on incidence of shark attacks. Now, clearly there is no association; buying ice cream in no way affects ...
• 26.4k

Bayesian criticism of frequentist p-value

The point that the authors are trying to make is a subtle one: they see it as a failure of NHST that, as $n$ gets arbitrarily large, the $p$-value doesn't tend to 1. It's a bit surprising that this ...
• 53.2k
Accepted

Frequentist perspective of regression coefficients and significance (coming from Bayesian background)?

A p value is the probability of observing a test statistic as or more extreme than the researcher's own test statistic, assuming the null hypothesis, and an assumed distribution model are both true. ...
• 26.6k

(Sudden deafness ended?) How can I better communicate -and did I find- a major (conclusion-flipping) statistical error in a recently published paper?

It's exciting to think that one has seen a fatal flaw in some published work. But sometimes that vision of a "fatal flaw" is a mirage, arising from a misunderstanding of the question at hand,...
• 63.8k

Hypothesis testing: why $\mu > 0$ (or even $\mu > \epsilon$) "seems easier” to substantiate than $\mu \neq 0$?

There are some misconceptions in your question that I need to clear up before getting to the answer. The null hypothesis $H_0$ in a statistical test is always the claim you want to argue against. The ...
• 31.4k
Accepted

What can cause exploding statistic values and p-values near zero with the Wilcoxon-Mann-Whitney test?

Nothing out of the ordinary is going on from the sound of it. In almost all cases, I get huge values for the statistic Have you looked at the range of possible values for the statistic? For the ...
• 261k
Accepted

Comparing model evaluations of machine learning and statistics

Is it true to some extent that statisticians are usually more concerned about the model's goodness-of-fit and the corresponding metrics of significance, and not that much about model's generalization ...
• 587

(Sudden deafness ended?) How can I better communicate -and did I find- a major (conclusion-flipping) statistical error in a recently published paper?

I write in respect to *. This study should be withdrawn. I'm not going to wade through the material here to form a view on whether you are right or wrong in your identification of an error. However, ...
• 96.6k

Correct use of Chi-square and hypothesis testing?

All of your concerns are valid and well articulated. The chi-square test simply provides the weight of the evidence that there is an association between attendance and purchase. Any causality is an ...
• 2,470

Bayesian criticism of frequentist p-value

I think you’re conflating two different arguments against p-values. Let me spell them out. By definition, p is distributed uniformly under the null hypothesis (or as uniformly as possible in discrete ...
• 1,254
Accepted

Is it still necessary to correct for multiple comparisons/testing if the sample sizes are large?

Think buying hundreds of fair dice. You do not know they are, though, and hence test if each has an expected value of 3.5 points, via throwing each many times (1000+). One of them must come up as &...
• 26.7k

What is the history of $p < 0.05$ or 95% confidence?

Fisher suggested the 0.05 level indirectly. He mentioned that two standard deviations is an easy rule for significance, and the 0.05 level is what approximately corresponds to it. From Fisher's 1925 '...
• 48.1k

Statistically compare two large continuous datasets

One feature (not a bug) of hypothesis testing is that it gets more sensitive to small differences as the sample size increases. Consequently, hypothesis testing considers more than just effect size, ...
• 31.7k

Can an irrelevant variable be significant in a regression model?

"Relevant" and "irrelevant" are not well-defined statistical terms that everyone agrees on. Case in point: the answer by @Demetri Pananos interprets "relevant" as "...
• 2,511

Can an irrelevant variable be significant in a regression model?

I think a variable can be irrelevant and significant at the same time. But, how do I explain that? This can be explained by using the concept of type I errors. Below is an example by repeating a t-...
• 48.1k

Likelihood of Global Catastrophe - Surely it cant be 80%?

Check the reference for the figure mentioned by the report: Sandberg, A. & Bostrom, N. (2008): “Global Catastrophic Risks Survey”, Technical. Report #2008-1, Future of Humanity Institute, Oxford ...
• 114k
Accepted

What is the concepts of nominal and actual significance level?

Note: I suspect that there are at least two different meanings of "actual significance level" around, but here's one that makes sense to me: The nominal significance level is the ...
• 12.3k
Accepted

What is the parameter in Spearman's $\rho$

Suppose $(X_1,Y_1),(X_2,Y_2),\ldots,(X_n,Y_n)$ are i.i.d random vectors with a continuous distribution. Let $R_i =\operatorname{Rank}(X_i)$ among $X_1,X_2,\ldots,X_n$ and $Q_i=\operatorname{Rank}(Y_i)$...
• 9,170

Comparing model evaluations of machine learning and statistics

The answer is no, too many claims in your post, not only to the main question. The difference between ML and stats is arbitrary, superficial, and not important. There is plenty of statisticians that ...
• 6,156

What is the exact mathematical relationship between the p-values and t-stat/degrees of freedom?

Accurate p-values for t-tests are not really feasible "by hand"; note that you can work out p-values from the cdf, so I will focus on that. At very low, integer degrees of freedom, the cdf ...
• 261k
Accepted

Can dropping an insignificant factor from a model make the model worse?

In this case you are relying on the wrong test to decide that Zone is not significant. Note that the coefficients of the Zone effect are large (>30) with huge standard errors. This happens when the ...
• 10.3k

How to report a P-value?

When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-significant P-values)? As a general guideline you want to convey as much information about ...
• 48.1k
Accepted

Linear relationship seems to depends from one value: would bootstraping help?

First off: If I extend this argument, would removing other value also impact the relationship? Almost certainly yes. The regression line slope depends on all observations, so removing any point will ...
• 96.8k

Regarding p-values, why 1% and 5%? Why not 6% or 10%?

A recent article sheds some light on the arbitrariness of $p$-values; the selection of two thresholds was motivated, at least in part, as a work-around to a dispute over publishing rights. Briefly, ...
• 79.4k

Bayesian criticism of frequentist p-value

For me a core issue here is that the Bayesian criticism of the p-value is based on Bayesian reasoning that a frequentist would not normally accept. For the Bayesian, the "true parameter" is ...
• 12.3k
Accepted

T-tests provide info about the margin of mean difference?

if I can use the t-test to claim that one dataset provides significantly better results than the other I wanted to point out that the expression "statistically significant" has been ...
• 3,110

Hypothesis testing: why $\mu > 0$ (or even $\mu > \epsilon$) "seems easier” to substantiate than $\mu \neq 0$?

Hypothesis testing: why $\mu > 0$ (or even $\mu > \epsilon$) "seems easier” to substantiate than $\mu \neq 0$? It seems easier because the one-sided t-test and two-sided t-test have ...
• 48.1k
Hypothesis testing: why $\mu > 0$ (or even $\mu > \epsilon$) "seems easier” to substantiate than $\mu \neq 0$?
the data may strongly support $\mu > 0$, but does not constitute sufficient evidence for $\mu \neq 0$ I don't know if this reasoning helps or if it is 100% correct... You accept to be wrong \$\...