Skip to main content

New answers tagged

0 votes

Can p-values for Pearson's correlation test be computed just from correlation coefficient and sample size?

Yes, the p-value can be computed from the sample correlation and sample size There are various versions of the Pearson correlation test that use different test statistics based on different ...
Ben's user avatar
  • 129k
10 votes
Accepted

Related low p-values that do not meet statistically significant thresholds

Interesting question, let me break it down into a few distinct problems: Do multiple borderline significant results indicate overall significance? The answer to this one is simple. No. If anything, ...
Frans Rodenburg's user avatar
1 vote
Accepted

Test whether my new method's type I error rate correctly matches chosen alpha

Empirical validation of null-rejection methods tldr: in null-rejection testing, the empirical rate is typically different than the nominal rate (due to approximations), but it only matters if it is ...
Guillaume Dehaene's user avatar
0 votes

Test whether my new method's type I error rate correctly matches chosen alpha

This is not an answer about statistical methods, but only to point out that it is common practice in the literature to not actually test whether a rejection rate is significantly different than the ...
A Friendly Fish's user avatar
4 votes

Are there alternative statistics to a p-value in NHST?

Your last statement talks about the probability of a hypothesis being true. That can be approached by Bayesian statistics. But that is a whole topic that I will not get into here (but is a good ...
Greg Snow's user avatar
  • 52.5k
4 votes

Are there alternative statistics to a p-value in NHST?

To put it briefly, this is now an operational question. Your observations are sound. In fact, nearly every statistician has (or should) consider the exact problem as you've described it. Closely tied ...
AdamO's user avatar
  • 63.7k
2 votes
Accepted

Testing for difference in trends of binary-parameter proportion/fraction versus continuous parameter between two (or more) different populations

Yes, you could do this by comparing the slopes of a spline for $M$, using a logistic regression model. Here is an example with some made up data with random trends: ...
Frans Rodenburg's user avatar
3 votes

Comparison of two test metrics

You need to precisely define "better". Once you do this, the solution is probably simple. If "better" means "more correlated with delta" then do the correlations. If &...
Peter Flom's user avatar
  • 125k
5 votes

Variable is significant in multiple linear regression but not in t-test of the subgroups

This question has multiple issues Location of the intercept One issue is that the main terms can be significant or not depending on the place of the intercept. This relates to R: Why does type III ...
Sextus Empiricus's user avatar
0 votes

Ideal test/p-value calculation for difference in means with small sample size and right skewed data?

I would look at how many of your control group are greater than both the comparison group observations / how many are less than both. Which one to use as your statistic depends on which tail you're ...
Acccumulation's user avatar
3 votes

Variable is significant in multiple linear regression but not in t-test of the subgroups

Looking at the results and the boxplots, dogs have clearly larger values than cats. Looking at the difference between tasks A and B, values for task B look lower for cats but slightly larger for dogs (...
Christian Hennig's user avatar
8 votes
Accepted

Variable is significant in multiple linear regression but not in t-test of the subgroups

You wrote: I am getting different p-values for a variable in t-test and its coefficient in multiple linear regression so I am unsure which one to believe. Believe them all, as long as the ...
Peter Flom's user avatar
  • 125k
2 votes

Variable is significant in multiple linear regression but not in t-test of the subgroups

(a1 and m1 are the same model.) What you're observing is that multiple regression is a more powerful approach than separate $t$-...
Frans Rodenburg's user avatar
8 votes

Ideal test/p-value calculation for difference in means with small sample size and right skewed data?

The limiting factor is the smaller of the two sample sizes. Because the power is exceptionally low in your situation, it is misleading to use p-values, as large p-values have no interpretations. If ...
Frank Harrell's user avatar
9 votes

Ideal test/p-value calculation for difference in means with small sample size and right skewed data?

Honestly, trying to do statistics on a sample of size 2 is like being "up the creek, w/o a paddle". This is not, and never will be, a representative sample of any population, and any ...
jginestet's user avatar
  • 1,805
7 votes

Ideal test/p-value calculation for difference in means with small sample size and right skewed data?

Ideal test/p-value calculation for difference in means with small sample size and right skewed data? There's really no "ideal". Normally you want two things: a significance level close to ...
Glen_b's user avatar
  • 286k
7 votes
Accepted

Estimating effects in the presence of a mediator

Method 1 My answer only addresses the statistical part of your question (Peter notes some issues with your model from a theoretical perspective, which is equally important). If you have a strong a ...
Shawn Hemelstrand's user avatar

Top 50 recent answers are included