# Tag Info

### Calculating statistical significance of gender discrimination in a CS class

As well as the issues already raised in the comments and the answer - if you were motivated to conduct this test just because you noticed the gender difference then you are HARKing (hypothesizing ...

### Proof that Sargan test statistic is distributed $\chi^2$

I will use your notation and write $M_A = Id - P_A = Id - A^T (A^T A)^{-1} A$ for the projection onto the orthogonal complement of $A$. Let's assume all k regressors in $X$ are endogenous, i.e., $k=m$....

### Calculating statistical significance of gender discrimination in a CS class

Your calculations are correct, as long as assumptions are correct (which may be not the case, as noted by Greg Snow in the comment section). However, one important problem is that this is quite a ...

### Common ways of p-hacking in scientific literature?

A typical approach to answer such a question is to look it up at Wikipedia which has a broad article on this topic https://en.m.wikipedia.org/wiki/Data_dredging following the links in that article, ...
Accepted

### Hypothesis testing in moderation analysis after Propensity Score Matching

What the code does is compute the log risk ratio (LRR) of the treatment for each level of the moderator (male). The first call (...
Accepted

### Why does the score test work for values longer in the tail that have a small log-likelihood derivative?

Remember that we're differentiating the log-likelihood. If $L(\theta;x)$ goes to 0 as $\theta$ moves away from the peak, then $\log L(\theta;x)$ will go to $-\infty$. Try sketching the log-likelihood ...

### Comparing pre and post treatment data - best statistical test?

I typically reference one of these. Save the ol' brain power for the programs.
1 vote

### Tests that Quantify Deviation from Null Hypotheses

I don't think you want a test at all. A statistical hypothesis test tells you whether you should a) Reject the null or b) Not reject the null. That's all. That's just what tests do. And the results ...

### How could Lilliefors use Monte Carlo if the estimand is not distribution-free?

It might not be distribution-free but for a given sample size it is location and scale free. The internally standardized joint distribution of the observations does depend on the form of the ...

### What is the best test to use?

My advice for beginning your explorations: Forget about statistical tests, and graph the data in various ways with as much detail as is reasonable.

### Best test for comparison of temporal counts

It very much depends on what exactly you mean by "closely related". One fairly obvious choice is correlation. This can be dangerous with time-series type data and lead to odd findings (e.g. ...

### Understanding the significance level of a confidence interval

It just indicates that if you repeat your experiment 100 times (e.g., draw random samples of size n from the population and construct the ...

### Is there a multivariate two-sample Kolmogorov–Smirnov test?

For a practical application of the multivariate KS distance in high dimensions, see my article New Python Library to Evaluate AI-generated Data and Compare Models. I actually created a Python library ...
1 vote

### Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?

While most of existing answers pointed out the multicollinearity is one common cause of this paradox, it seems that none of them demonstrated why it is so from the mathematical perspective. This ...
1 vote

### Two sample proportions: How to test the hypothesis that the difference in proportions is less than a specified threshold?

You say that your null is that the difference in conversion is at least 1 percentage point, but in the comments you also agree that you suspect the change is at least one percentage point. Those two ...

### Transitivity of Wilcoxon signed-rank test

Does this meet the conditions of a counterexample? Or have I misunderstood something somewhere? ...
Accepted

### Transitivity of Wilcoxon signed-rank test

Update: I'm no longer convinced that the ordering is transitive, but it is still described by the pseudomedians. It's just that when you don't have a location shift the pseudomedians need not be ...
1 vote

### Does the term “bootstrap” comprise Monte Carlo samples of null models a.k.a. “surrogates”?

For whatever it’s worth, I stumbled upon a publication that is essentially about this, namely namely Jason H. Moore, Phys. Med. Biol. 44, L11 (1999) (letter to the editor). The author notes that the ...
1 vote

### How to do a pairwise test of each regression model coefficient over groups?

You can add the binary grouping variable (G) as a predictor to your model together with its product with each predictor variable X (moderated regression), for example, G*X. The regression slope ...
1 vote

### How do we know the distribution of regression coefficients

The Central Limit Theorem gives, as is correctly stated in the question, the asymptotic normal distribution of the suitably standardised sample mean. This statement can not only be used to say ...

### How do we know the distribution of regression coefficients

The estimated regression coefficients are functions of the data (both the response and explanatory variables). The regression model specifies the conditional distribution of the response given the ...

### When failing to reject a null hypothesis can we also calculate the probability of observing the treatment assuming the alternative is true?

Yes we can, and in fact, this calculation is just a variation on the power function. Suppose you have a test using the parameter $\theta \in \Theta$ and with significance level $\alpha$. Then the ...

### Anova with a specific reference group? Dunnett's test overall p-value?

I say the overall test you want is the usual F-test. You say you do not care if two treatment groups have different means. However, if two treatment groups have different means, then at least one of ...

### Anova with a specific reference group? Dunnett's test overall p-value?

For your second question, yes, it is possible and it illustrates the issues with relying to "sig." vs. "not sig." This is not something to rely on. But don't take my word for it, ...

### Why do we adjust for within-group variability in statistical testing of differences in group means?

Stephen gives a very good answer (+1). I wrote an entirely non-mathematical answer on Medium. Why analyze variances to compare means which may appeal to the less mathematical among your students. (...
Accepted

### Why do we adjust for within-group variability in statistical testing of differences in group means?

The issue is that you will always have differences between the responses observed in your two groups, simply because of random variations. Importantly, you will have such differences even if your ...
Accepted

### Is a power analysis appropriate when the predictor variable is bounded between 0 and 1?

Yes, power analysis is absolutely valid in this case. It will work for any kind of predictor. After all, most predictors are bounded somehow by nature, and some can only take the values 0 and 1 (e.g., ...

### Am I understanding critical value and p-value correctly?

An alternative is to forget about p-values and critical values. Instead, compute the difference between the two means, and the confidence interval for that difference. Then interpret the confidence ...

### A specific example of two-sided chi-squared test

An old question, but for completeness: Based on my understanding of your question, you're simply specifying an $\alpha=0.10$ error rate, and promoting that if one specifies this error rate (through a ...

### Practical significance, especially with percents: "standard" measure and threshold

To add to most other answers, as you were looking for references, below you'll find some saying that you should avoid using arbitrary thresholds. Note that there are various standardized effect sizes ...

### Infinite upper confidence interval using Fisher exact test in R

Alternatively, I have created a bootstrap distribution for the odds ratio to get the confidence intervals (using the infer package in R), but I am not sure this is a valid approach. Bootstrapping is ...

### Infinite upper confidence interval using Fisher exact test in R

There's a one-sided confidence interval because you asked for a one-sided test

### Detecting biased coins

The question asks us to distinguish between the possibilities $p = 0.5$ (unbiased) and $p = 0.57$ (biased) ("confirm... if the coin is actually biased or not, with a particular bias of p=0.57&...

### Computing a p-value of a 2-sided Chi-squared test for one variance

For a two tail test the p value has to be doubled. As the maximum value of p can be 1, we should take the value which after doubling will be less than 1. That is to say, right tail or left tail area ...