6
Think buying hundreds of fair dice. You do not know they are, though, and hence test if each has an expected value of 3.5 points, via throwing each many times (1000+). One of them must come up as "best", and if you do not account for multiple testing, almost certainly statistically significantly so.
Recall that the probability that a true null is ...
3
The Spearman correlation is not a linear correlation of the data, but a linear correlation of a transformed version of the data -- specifically, the correlation of the rank-transformed data. It doesn't "see" what you show in your plots. (The second plot has been removed from the question, but I will leave it in my answer as it still serves to ...
2
A easy way to make sense of this is in terms of effect sizes, Type 1 errors, Type 2 errors, and Power.
Let's say that you're looking at correlations, and you have $N$ data points.
Your effect size is the correlation coefficient, $r$.
Your Type 1 error rate, $\alpha$, is the probability of concluding that there is an effect, $r \neq 0$, when there really is ...
2
You may want to use a prediction interval when
considering a new patient, who was not part of the previous
analysis for the control group.
Suppose the true scores for the control group are
distributed $\mathsf{Norm}(\mu =50, \sigma=7).$ In
a real application you could not know the population
mean and standard deviation, but would estimate
them using a sample ...
2
Sampling without replacement can be modelled using the hypergeometric distribution. Using the notation from the Wikipedia article, denote $N$ the population size (326,000), $K$ the number of males in the population (240,000), $n$ the number of draws (197,395) and $k$ the number of observed males in the sample (146,862).
Using the upper tail of the CDF of the ...
1
Hypothesis testing is not a means of performing sensitivity analyses. Looking at a range of $p$-values does not give you an idea about what a possible significance level should be.
You need to use graphical tools, like a histogram and QQ-plot, to look at the distribution of residuals. Since the main test is an ANOVA, the simple box-and-whisker plot of ...
1
Their suggestion was that we don't have to worry about [multiple testing], because the sample size of each test is going to be big enough (we're looking at minimum of n=100 but frequently n=1000+).
Here's a scenario where your colleagues would be right in practice even if wrong in theory.
Your many tests fall neatly into two categories: $H_0$ is correct (or ...
1
In general, when you don't get an answer on this site, it is mainly because people are struggling to understand the question... while you might have a valid point, which, I think, is the case here.
Most of the time, providing data sample contributes a lot to understand better your challenge. In your comment, you indicated that the data looks like this:
year
...
1
You would pass in the sum of your observations, and set the T parameter in poisson.test to the number of days you have taken samples:
#pretend these are your data points
x <- 8:16
poisson.test(sum(x),T=length(x),r=10)
# > Exact Poisson test
# >
# > data: sum(x) time base: length(x)
# > number of events = 108, time base = 9, p-value =...
1
I have seen some papers that are removing f from test data but not the
training data. In that case, say a model m trained on the original
train data has a performance of p" on the modified test data and p" is
still not significantly different from p. From that, they are saying
that f is not important for m to solve C. Is this a valid conclusion?
...
Only top voted, non community-wiki answers of a minimum length are eligible