5
Please try not to be too concerned with p-values. They don't tell you anything about practical significance.
I am wondering if the interpretation is as simply as, for example, there is a significant difference in the maximum depth (max_depths) reached between feeding and non-feeding dives, with maximum depth taking higher (deeper) values in feeding dives.
...
5
Without seeing the data and the model it is hard to be absolutely sure, but I would think that it just means that the p value is extremely small. We can do a simulation to show this quite easily:
N <- 1000
group <- rep(c(1,2,3,4,5,6,7,8,9,10),100)
x <- seq(1:N)
y <- rnorm(N, 100, 1) + x + group/10
group <- as.factor(group)
lmm.lme <- lme(...
4
Warning pessimistic/cynical post
We do not want to move away from significance.
That is a false premises that lead to your question.
Recently I have found that many statisticians are speaking of moving away from significance.
..
Since we want to move away from significance...
We do not want to move away from significance. Significance is important. It is ...
4
To construct a level $\alpha$ rejection region we first calculate the level $\alpha$ critical value $c_\alpha$. For a two-tailed
test based on a test statistic that is $N(0,1)$ under $H_0$, the critical value is defined implicitly by
\begin{equation}\label{deficritval}\tag{1}
1-\alpha/2=\Phi(c_\alpha)
\end{equation}
where $\Phi$ denotes the standard normal ...
3
This is too long to be a comment, but just to add to the excellent answer by Sextus, one issue with "significance" is the arbitrary nature of the significance level(s). Often these are dictated by whatever is common practice in a particular field.
Also, when a researcher performs a test and finds a p-value of, say 0.0499999 they may claim to have ...
3
In general p-values are not equal to zero, but are just very very tiny, so statistical software tends to report 0 or 0.000 instead of a number like 0.0000000000183 (I tend to report such like so: "p <0.001").
Given that your sample size is in the thousands, and that in your specific case you have (fixed effect) coefficients with t test ...
3
Looking at the code for kendall.ci, it seems to be using a U-statistic formula for the variance, which will be correct only for continuous distributions (though it should be an ok approximation more generally).Since you have a lot of ties and the $p$-value only just fails to match the confidence interval, I think that's the issue.
According to the ...
3
I would say "none of the above". 1 2 and 3 are definitely wrong and 4 seems to be some sort of badly stated "arose by chance" thing, but it doesn't mention that the null has to be true and it doesn't define "noise". The Wikipedia definition is OK, but it should mention that the sample has to be randomly selected.
2
dat$mean_diff_pval <- pt(dat$mean_diff_tstat, dat$mean_diff_dof)
This is wrong. The p-value is $P(|t|>|t^*|)$ and you can calculate it by:
pt(abs(dat$mean_diff_tstat), dat$mean_diff_dof, lower.tail=FALSE) # P(t > |tstat|)
+ pt(-abs(dat$mean_diff_tstat), dat$mean_diff_dof) # P(t < -|tstat|)
or
dat$mean_diff_pval <- 2*pt(-abs(...
2
If your group differences are sufficiently blatant, very small $p$ values can easily result. For instance:
Here, we get $p\approx 2.3\times 10^{-15}$. ("$\approx$" because you shouldn't take this number too seriously.) R code:
set.seed(1) # for reproducibility
group_1 <- rnorm(120)
group_2 <- rnorm(60,3,2)
t.test(group_1,group_2,var.equal=...
2
The false discovery rate (FDR) is the expected proportion of type I errors, which is where you incorrectly reject the null hypothesis.
α is the p-value you choose to use. You have defined it as 0.05, which is a common choice. At this p-value, there's a 5% chance of a Type 1 error.
1
You can use the Neyman-Pearson lemma to determine the most powerful test to apply.
First, we want to move into testing two simple hypotheses:
$$
H_0: \quad \alpha=0\\
H_1: \quad \alpha=\hat{\alpha}
$$
where $\hat{\alpha} \in \{ \alpha : \alpha > 0 \}$. Please bear in mind that $\hat{\alpha}$ is a fixed value, so we have the full distribution defined. We ...
1
There are many help sites on the Internet for doing two-factor ANOVAs.
I will leave that part to you. The rest of this is in case you find that
residuals of your ANOVA are not normal (or differ widely in variance among
the four treatment combinations).
Below is an example of two (simulated) bimodal samples, and a permutation tests
to see if their means ...
1
Your first approach will be fine; see for example here for more background. The KS test works for unbalanced data sets.
(And yes, you ought never to average p-values, at least not by taking an arithmetic mean. If you do decide to run multiple draws, consider Fisher's method. But you don't need the sampling approach.)
1
The Bonferroni correction (or its more powerful but equally sensitive Holm modification) controls the family-wise error rate, the chance that any of your nominally significant results is a false positive. If that's what you want to control, then you need to consider all 40,000 comparisons. You could instead try to limit the false-discovery rate, the fraction ...
1
You need to correct for all of the comparisons you are doing. So if that's 300,000,000 comparisons you need to correct for that many multiple comparisons.
But consider what some standard corrections for false-discovery rates (FDR) and family-wise error rates (FWER) protect you from. Say you have data in which there are no true associations but you do a lot ...
1
There may be an issue on the cycle length of the random number generator, which could be an issue for large n!
Apparently, cycles repeat at some point.
Try adding a random choice to which of two (or more) possible new random #s you will select. If a repeating cycle issue, this will add randomization.
See if you can detect any improvement.
1
In my opinion you have to perform a one-sample Kolmogorov-Smirnov Test. You have a sample of the random variable Y and you want to check if the random variable is two-parameter APE distributed. So I've updated your cdf such that it has a further argument for the values of the sample, called argument y.
However, you have some problems performing the KS-Test. ...
1
With 0.21, your sample odds ratio is far below 1, so there is absolutely no evidence that the true odds ratio is above 1 (H1). A p value of 1 makes perfect sense.
Maybe you mixed up the specification of the alternative hypothesis, but statistics can't help you out with this. Of key importance are the following questions, which you should be able to answer ...
1
The distribution of the p-value under the null hypothesis can tell you whether the test achieves its actual size or not. If, say, less than 5% of the distribution falls under the 0.05 mark, the test is said to be anticonservative (rejects too often), and it's conservative for more than 5%. Fisher's Exact Test is a commonly accepted conservative test. P-under-...
1
Bayesian p-values are normally used when one would like to check how a model fits the data. That is, given a model $M$ we wish to examine how well it fits the observed data $x_{obs}$ based on a statistic $T$, which measures the goodness of fit of data and model. For this, suppose we have a model $M$ with probability density function $f(x|\theta)$ and with ...
Only top voted, non community-wiki answers of a minimum length are eligible
