New answers tagged statistical-significance
0
votes
On choice of y-axis range for visualization
First recommendation is to include 0, which is what your 3 proposals do.
Second one is to absolutely show the CI's on your plot; not showing the CI's is hiding the uncertainty on your prevalence ...
- 3,370
1
vote
On choice of y-axis range for visualization
A y-axis range that is just large enough to show the entire range of the confidence intervals over time would be most useful for your audience.
With properly labeled axes there is no "propaganda&...
- 98k
-1
votes
K-S test is too sensitive
I faced with the same issue. I did not want the K-S test to be too sensitive to the central tendencies and the standard deviation of the samples -- but more of the overall shape of the distribution.
I ...
1
vote
How can I determine significant differences within factors and interaction effects?
Make sure that you look at the dose-response curves first. That can save a lot of trouble later on.
Your sense is correct that you shouldn't be doing multiple t-tests in a situation like this. There's ...
- 98k
0
votes
MCMC direct comparison of difference of two parameters
One approach is to use domain knowledge to define a region of practical equivalence (ROPE) around the null value, and then compare the posterior distribution of $\beta_1 - \beta_2$ to that ROPE. If, ...
- 131
4
votes
Is random slope meaningful when the relevant fixed effect is not significant?
The p-value isn't the major concern here...its the perfect negative correlation of random effects, which is a major sign the model is wrong. As you noted, there are miniscule differences in random ...
- 16.2k
1
vote
Why isn't a confidence level of anything >50% "good enough"?
A quote from the image you linked in the comments:
Based on the total number of Impressions and Clicks, we are 84.67% sure that your Variation will perform better than the Original.
This wording is ...
- 587
4
votes
Why isn't a confidence level of anything >50% "good enough"?
First, let me say that the fact that "The test will run and collect data until it can make a decision at 95% confidence" is fundamentally flawed and will never give you a valid result. I ...
- 3,370
2
votes
Why isn't a confidence level of anything >50% "good enough"?
I'm an historian of philosophy, and as such, fairly well-read in the realm of epistemology. Evidence, Proof, Truth, Justification are all matters of epistemology, so I may have something useful to add ...
23
votes
Accepted
Why isn't a confidence level of anything >50% "good enough"?
You are totally correct that there is nothing "special" about the standard of 95% confidence, which is in many ways just a sort of historical artifact: statisticians have been using "95%...
- 5,417
4
votes
Why isn't a confidence level of anything >50% "good enough"?
Your wording is ambiguous, but 'confidence' is usually used in statistics as part of 'confidence interval', which is a way to communicate our level of uncertainty in a parameter. For historical ...
- 19.8k
4
votes
Why isn't a confidence level of anything >50% "good enough"?
Your question is quite topical for me, as I had just been making my own enquiries after coming across this terminology. Don't be concerned you don't know what it means (I'm glad that it is not being ...
- 830
3
votes
How to Interpret Statistically Non-Significant Estimates and Rule Out Large Effects?
Also agreeing with Christian Hennig's advice on equivalence tests (+1).
Just pointing out here that you may not need to run any new test, compute any new p-value, or any such. You may very well ...
- 3,370
9
votes
How to Interpret Statistically Non-Significant Estimates and Rule Out Large Effects?
I agree with Christian Hennig's advice on equivalence tests (+1).
In addition, I think it can be useful to do a power analysis for the minimum effect size that would be interesting/meaningful in your ...
- 19.8k
10
votes
How to Interpret Statistically Non-Significant Estimates and Rule Out Large Effects?
One thing that you may want to look at are equivalence tests (TOST). Here the standard way of running tests is reversed and you test the null hypothesis that an effect has a certain minimum size ...
- 26.9k
2
votes
Accepted
Testing for significance where results are expressed as a ratio to an in group control
A better way to approach this is to model all the growth rates in terms of both Strain and Treatment via multiple linear ...
- 98k
0
votes
Accepted
How do I test if a population change is statistically significant?
I think, rather than testing for significance, you can restate your goal maybe to forecast the number of weddings for each parish. I would suggest a model with a ...
- 3,952
0
votes
How do I identify common features/similarities of members in a class?
I think the other user provided a great response since you were interested in specific features of those in 1 class, etc.
But since you asked about "similarity", maybe this is of interest ...
- 1,123
0
votes
Ceiling effect in mediation AND/OR moderation
I don't think that having the results in hand and rewiring a post-hoc solution to the problem should drive how you construct models. You were already testing a model, now you have to deal with that ...
- 16.2k
1
vote
What does getting a non-significant log(theta) mean?
tl;dr it's probably not anything you care about.
Coefficient tables like this are always (in my experience) testing the estimated coefficient against a null value of zero. In this case, since we are ...
- 45.2k
0
votes
Measure the significance of differences in internal distances in two groups
I am afraid that the fundamental issue with your approach is that you are using proportions, rather than counts (there is one possible but unlikely exception, which I will cover later). I will assume ...
- 3,370
0
votes
Rejection threshold of the Benjamini-Hochberg procedure
The BH-FDR procedure calls significant R of the m tests where
R_m = sup{j : P_(j:m) <= alpha j/m }
where P_(j:m) are the order statistics.
Once R_m is defined, the following expression is ...
- 81
1
vote
How to adjust a variable by age, sex and BMI
You don't adjust the outcome (cognitive score) values themselves. You include the other variables (age, sex and BMI) along with the main predictor of interest (level of diabetes) into a multiple ...
- 98k
4
votes
Asymptotic consistency of the Clopper-Pearson interval
Yes, it is true. From the way Clopper & Pearson define the interval $(p_L, p_U)$, the conservatism when seeing data $X=x$ is bounded above by $P(X=x; p=p_U)+P(X=x; p=p_L)$, and this is $O(n^{-1/2})...
- 43.6k
0
votes
How to know what effect size is excluded by a negative trial
Going back to yourt 2 possibilities:
1. There really is no difference
2. We have lost a potentially clinically significant difference due to a lack of power
It really is not quite that; in fact it is ...
- 3,370
0
votes
Accepted
How to know what effect size is excluded by a negative trial
what can we say about the effect size we have actually excluded? What remains plausible?
This link to Frank Harrell and others talking about how to interpret results from treatment effects is super ...
- 38.6k
0
votes
advice on statistical method
For ANOVA, you need only 1 treatment, with more than 2 levels (1-way ANOVA), so you may have confused treatments, and levels. In your case, you have 2 treatments, each with 2 levels, and a 2-way ANOVA ...
- 3,370
Top 50 recent answers are included