New answers tagged

8 votes

How does reducing my alpha-threshold to 0.005 affect my required sample size?

As you wrote ".. there is no "simple" function mapping the old size to the new one .." --this is what I expected. However, following your suggestion and then plotting new against ...
PuzzledBiologist's user avatar
7 votes
Accepted

How does reducing my alpha-threshold to 0.005 affect my required sample size?

There is very likely no "simple" function mapping the old sample size to the new one. The way to go about this would be to calculate the effect size detectable at the "old" sample ...
Stephan Kolassa's user avatar
1 vote

Obtaining P value in LASSO regularized linear regression showing that the model is generalizable

It sounds like your supervisor wants to show p-values for coefficients similar to those that you would show for unpenalized regression. You can't just plug the selected predictors into an unpenalized ...
EdM's user avatar
  • 90.3k
5 votes

How do I interpret low subgroup interaction and high efficacy in only one group?

The Dangers of P-Value Hunts Peter already noted issues in your interpretation that I completely agree with (see my answer here, here, or here, which all show how meaningless $p$ values can become ...
Shawn Hemelstrand's user avatar
6 votes

How do I interpret low subgroup interaction and high efficacy in only one group?

Welcome to CV. There are a few errors in your analysis, or, rather, in your interpretation of your analysis. First, you tested interpretation and found it was not significant. You concluded that the &...
Peter Flom's user avatar
  • 117k
5 votes

Sample Size impact on Effect size

All things being equal, the major difference is the variation in the effect as a function of sample size. A very simple simulation in R shows this as the case. Here I simulate two groups who have the ...
Shawn Hemelstrand's user avatar
4 votes

Sample Size impact on Effect size

The question is fully addressed by meta-analysis (see, e.g., Introduction to Meta‐Analysis by Borenstein et al.), which includes studying of how the effect size vary from one study to another. Indeed, ...
Roger V.'s user avatar
  • 3,863
2 votes

Report standard errors and exact p-value

The $p$ values are already a hairy subject, but I would trust them much less if there were no indicators of where that $p$ value came from. From my personal standpoint, you should always: List the ...
Shawn Hemelstrand's user avatar
6 votes

Report standard errors and exact p-value

This is a style issue, and each journal has its own. However, if it was up to me, I'd require standard errors and make p values optional.
Peter Flom's user avatar
  • 117k
13 votes

Sample Size impact on Effect size

When your sample size is small, there will be more sampling error attached to your effect size estimates and hence more uncertainty/wider confidence intervals. In that sense, I wouldn't say the effect ...
Christian Geiser's user avatar
9 votes

Sample Size impact on Effect size

As long as the sample is random, the estimate of the effect size is unbiased even in small samples. However, other things being equal, it will be less precise. It's still your best guess, but the ...
Peter Flom's user avatar
  • 117k
0 votes

Never understood the concept of the p-value, it should be higher than 0.05

Sometimes it is easier to look at a visual representation to better understand the as extreme or more part of the definition of the p-value, or the p-value itself. Suppose you repeat an experiment ...
Stefan's user avatar
  • 5,896
3 votes

Never understood the concept of the p-value, it should be higher than 0.05

You wrote: If I understand correctly 'more extreme' are results in favour of the null hypothesis, because then a high p-value is in favour of it. this is just the opposite of the case. More extreme ...
Peter Flom's user avatar
  • 117k
0 votes

Interpretation of p-value in hypothesis testing

I’ve attempted to describe this as a distinction between assertion probabilities and decision errors here. Part of the issue is that essentially $\alpha = \Pr(p < 0.05 | H_{0})$ and that setting $\...
Frank Harrell's user avatar
4 votes

Never understood the concept of the p-value, it should be higher than 0.05

'More extreme results are in favor of the null hypothesis' --> This is not true! The point of the p-value is to quantify how extreme your observation is if you assume the null hypothesis to be true....
Mathemagician777's user avatar
2 votes

P-values extremely low for linear mixed effects model

If you have a large sample, even trivially small effects will be detected as significant. The p value calculation is almost certainly correct. It just very probably does not answer the question you ...
Stephan Kolassa's user avatar
3 votes

Why the p-values of a zero-inflation model are not useful, and how do we know statistically significance?

Why the p-values are not useful? As Ben says, the p-value corresponds to comparing the coefficient to a reference value of 0. A 0 on log odds scale corresponds to 50% probability of structural zero. ...
Alex J's user avatar
  • 1,796
0 votes

p-values from CIs?

The estimated distribution from a bootstrap method does not always give sufficient information to compute a p-value, and neither a confidence interval. One problem is that p-values (and confidence ...
Sextus Empiricus's user avatar
2 votes

p-values from CIs?

A CI can be constructed without a hypothesis. But a $p$-value requires you to state one. Consider, for one, that a difference in standard deviations is not a natural (interpretable) quantity but a ...
AdamO's user avatar
  • 61.9k
11 votes

p-values from CIs?

To compute a p-value with a Monte Carlo test, you must sample values under the null hypothesis and compute the fraction of simulation results that fall beyond the value for the statistic computed from ...
cdalitz's user avatar
  • 5,002
3 votes
Accepted

What is the "right" way to define "extreme" in a hypothesis test?

To see if the test is "good" you need to analyse the properties of the power function As you point out, it is possible to derive a test statistic ---and thereby derive the "evidentiary ...
Ben's user avatar
  • 123k
0 votes

What is the "right" way to define "extreme" in a hypothesis test?

Hypotesis testing Is about the value of some parameters and any parameter have his parameters space. Null hypotesis can be about a point in the parameters space or an interval. Alternative hypotesis ...
markowitz's user avatar
  • 5,314
3 votes

Can I use the ratio of two p-values under two hypotheses as a likelihood ratio?

It has been proposed at least a few times to use a plot of p-value vs the value null hypothesis parameter as a pseudo-likelihood like object (see for example The p-value Function and Statistical ...
Michael Lew's user avatar
  • 14.9k
3 votes

Seeking Papers with Significant Associations in Univariate but not in Multivariable Analysis Due to Sample Size

You are comparing two different things: Univariate analysis assesses marginal distribution, whereas multivariate analysis assesses conditional distribution. They answer different questions. Both ...
2 votes

Calculate SE of regression coefficients using p-value (for meta-analysis)

It is possible with that information. You need to backcalculate the value of $t$ corresponding to that $p$ with the relevant degrees of freedom. If you use R then qt(p/2, df) should do it. Then since ...
mdewey's user avatar
  • 17.7k
1 vote

Seeking Papers with Significant Associations in Univariate but not in Multivariable Analysis Due to Sample Size

It would be interesting to see a simulation meeting your criteria. Usually in practical settings, the more variables you have the more probable it is that you will find the relationship between ...

Top 50 recent answers are included