In hypothesis testing, the $p$-value refers to the probability of seeing a result as extreme (or more so) than the one observed, given that the null hypothesis is true. When the $p$-value is small, then there is good evidence that the null hypothesis is false, since it would make the observed data ...

learn more… | top users | synonyms (2)

4
votes
1answer
19 views

Understanding multiple KS tests

I have read these two questions Why are p-values uniformly distributed under the null hypothesis? and Understanding scipy Kolmogorov-Smirnov test And this inspired me the following experiment. I ...
0
votes
1answer
16 views

Combining multiple p-values

Is there any statistically relevant way to combine multiple p-values measuring different hypotheses but that are all trying to measure the same thing "in spirit"? Eg, if I wanted to measure some ...
8
votes
5answers
97 views

What is the relationship between $p$ values and Type I errors

In hypothesis testing we set an accepted level of Type I error probability $\alpha$ and observe whether a sample statistic is equally likely or less likely to be observed if the null hypothesis was ...
0
votes
1answer
38 views

Interpreting high P-Value in ordinal rank correlation measures

I'm currently evaluating a machine-learned ranking task. Two (among other) performance measures are Spearman's Rho and Kendall's Tau (rank or concordance correlation). I'm basically comparing a ...
2
votes
0answers
26 views

How is plot of p-values distribution related to calculation of the false discovery rate?

I'm running t-tests for all combinations of pairwise comparisons of average length of stay for patients among 30 hospitals. My first approach was to use a Bonferroni corrected significance level of ...
2
votes
0answers
46 views

Converting Chi Square Adjusted Residuals to p values

I am running a chi squared analysis in SPSS on a 5 x 2 contingency table. The overall chi square p value was < 0.0005, so I ran post hoc tests (adjusted standardized residuals on the individual ...
3
votes
0answers
49 views

How to calculate p-value using an algorithm on R assuming that distribution is unknown

For a given sample x, where x is distributed with a normal distribution mean known but variance unknown. I am testing the hypothesis that variance is equal to one or greater than 1. The question is to ...
3
votes
1answer
37 views

p-value correction for multiple t-tests?

My dataset consists of $n$ genes, each of them described by a vector of expression values, $5$ for "healthy" individuals, and $5$ for "unhealthy" individuals. I am going to run $n$ t-tests (one for ...
0
votes
0answers
26 views

understanding of p-value in multiple linear regression

Regarding the p-value of multiple linear regression analysis, the introduction from Minitab's website is shown below. The p-value for each term tests the null hypothesis that the coefficient is ...
0
votes
0answers
15 views
3
votes
0answers
17 views

Determining statistical significance in difference between two CRF models

I have trained two Conditional Random Field (CRF) models Ma and Mb in CRF++, the first using a simple set of features, the ...
2
votes
0answers
19 views

Stouffer's method in log space

I'm combining p-values using Stouffer's method in R using the code from the linked Wikipedia page. Since some p-values can be close to 0 the combined p-value becomes 0 as a result of underflow (I ...
0
votes
1answer
33 views

How to obtain p-values for a given Kendall's Tau value (not calculated from data)

I want to obtain p-values for Parker's Tau-U (Parker et al., 2011). which is a measure based on Kendall's Tau. Tau-U is related to Kendall's Tau in such a way that it's possible to translate a Tau-U ...
2
votes
0answers
35 views

Running sequential tests as data is collected - practical advice for researchers

I've observed that many experimental labs in my field don't do a power analysis before they start collecting data. In practice, it seems that many researchers collect data and periodically run ...
2
votes
1answer
44 views

How can I calculate p-values in R?

I am trying to construct a hypothesis test for the variance of two populations. My test statistic is $F=$${\sigma_b^2}$/${\sigma_A^2}$. I want to implement this as a function in R so I computed: ...
1
vote
0answers
22 views

Error propagation and p value

Say I have parameters $a(x$) and $b(x)$, with $N_a(x)$ and $N_b(x)$ experimental measurement counts, respectively. $x$ is an independent variable ($x_1, x_2, ...$). For each $a$ and $b$, I can ...
0
votes
1answer
128 views

Why is P value interpreted under null hypothesis? [duplicate]

P value is explained very nicely in the link . But still i have not understood Why is P value interpreted under null hypothesis?
0
votes
1answer
32 views

Permutation test and p-value

I construct a permutation test in order to see If two samples come from the same distribution or not. I have two vectors $x,y$ that hold values of sampled values from two populations and the test ...
0
votes
1answer
33 views

Excel: Computing a pvalue [closed]

An event has an expected probability of .4. After 100 trials, the event occurred 30 times. How can the pvalue be computed in Excel?
3
votes
1answer
50 views

sample size too large? [duplicate]

I always thought larger sample sizes were better. Then I read something somewhere about how when sample sizes are larger, it's easier to find significant p-values when they're not really there (i.e., ...
10
votes
2answers
397 views

p-value subtlety: greater-equal vs. greater

As I am reading Wassermann's book All of Statistics, I notice a fine subtlety in the definition of p-values, which I cannot make sense of. Informally, the Wassermann defines the p-value as [..] ...
1
vote
1answer
72 views

Distribution of p-values in this thought experiment?

I'm trying to check whether my clustering was informative above and beyond random clustering. This is my thought experiment to do it, can someone help? Suppose I have a large number, $N$, groups. ...
1
vote
1answer
46 views

Bootstrap glm and extract pvalue

I am running a glm model using bootstrap, I can extract the coefficient mean and the confidence intervals for all the factors in my model. But how can I get the pvalue from there? Model: ...
0
votes
0answers
11 views

Compute effect size based on simulation using different lists of different length

For my research purpose I wrote a small simulation to compute an empirial p-value based on an input gene list and a reference gene list. The idea is to check for each gene in the input gene list, the ...
1
vote
0answers
30 views

Tukey HSD, insignificant p-value when the confidence interval does not contain 0?

I am using TukeyHSD to do multiple comparisons. I am seeing strange results for some of the pairs: the adjusted p-value is greater than the specified significance level but the corresponding ...
2
votes
1answer
36 views

How do I interpret the p value returned in scipy's Mann Whitney U test?

I have a program in which I need to perform an action based on what results of a Mann Whitney U test are. I know the criteria that need to be met, but I would still like to understand what the results ...
2
votes
0answers
43 views

Implementation of Bayes posterior predictive check

I have a question concerning the implementation of a bayes posterior predictive check. Let us assume i have this model (implementation is in R and jags): ...
0
votes
0answers
20 views

how to report p-values from t-test and ks test in the same table for publication?

I want to show in a table that two samples are different in a number of attributes (or not different). I want to report the mean, st.err. and the stars at conventional values of p-value from a t-test ...
0
votes
0answers
47 views

Is reverse of p-value also uniformly distributed?

If p-value, $p(F|H_0)$, is uniformly distributed $U[0,1]$, so is true to say, that $p(F|H_1)$ is also uniformly distributed $U[0,1]$ or even more: is it true $p(F|H_0) = 1-p(F|H_1)$? I'm not sure if ...
3
votes
1answer
162 views

independent replicate experiments yielding contrasting results

This is a though, simple experiment, for thinking efficiently about a real problem much more complicated... Imagine a simple experiment, trying to answer a simple question. For example, is body ...
2
votes
2answers
48 views

Why do we compare p-value to significance level in hypothesis testing of mean?

I know that: p-value = P(observed or more extreme outcome of sample mean | Null hypothesis is true) But why do we compare it with significance level (alpha) while determining if it is high or low? ...
0
votes
0answers
40 views

Analysis of the Benjamini-Hochberg adjusted p-values

I have: Condition #1 : 5 independent objects (1 object = a time series) Condition #2 : 5 independent objects My GOAL: test H0 that using condition #1 gives different results from using condition ...
0
votes
0answers
26 views

p-values, prior probabilities

I've got a set of N normal independent normal distributions, each representing a signal. I also got a new data sample, a vector $v$ of size Nx1. Now let's say I compute the p-value using the ...
6
votes
2answers
178 views

R t.test … NOT significant anymore

I got very confused while looking at help examples of the t.test function ...
0
votes
1answer
66 views

How to calculate p value from distribution of test statistic

I was wondering how I can calculate p-value from a distribution of test statistic manually? Regards,
1
vote
0answers
31 views

p-value over q-value?

I am trying to figure out when (during multiple testing) to use $p$-values and when to use $q$-values. This student thesis explains how this particular theorem (Eq. 11 in the paper) allows one to ...
2
votes
0answers
18 views

Weighting p-values

In KEGG pathways (Kyoto Encyclopedia of Genes and Genomes) data set designed as columns and rows. The columns consisted of 50 pathways and the rows are 189 genes. The genes can work in certain ...
3
votes
1answer
76 views

Interpreting results of chi-squared

Situation: A/B test of a single website change on a landing page. Alternating visitors to the landing page are shown a variation. The goal is to convert more visitors into customers. With data that ...
0
votes
0answers
27 views

P value distribution skew & hypothesis testing

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
1
vote
0answers
98 views

GLMER: Conflicting results from p values and bootstrapped confidence intervals

Please don’t be shocked by the amount of code in this post, this problem is (hopefully) not as complex as it seems. This dataset ...
0
votes
1answer
107 views

R lme4 1.1-7 glmer: p values for fixed effects in summary() and likelihood ratio test comparison

I'm using glmer() with a binomial response variable. My optimal model has two fixed effects (flow and DNA) which in summary() show a non-significant p value but ...
0
votes
2answers
48 views

p-value test for comparing classifiers

I have to use p-value test to compare two classifiers. I am getting the error vectors of 2 hypothesis and I am using their difference as an input in ttest function of MATLAB ...
3
votes
1answer
25 views

Do I need to do p-value correction after Kendall correlations?

I'm a neuroscientist and I've done some statistics before, but I'm fairly new to corrections of the p-value (like Bonferroni). I've got 2 groups of patients. For each group, I look into 3 areas of ...
2
votes
0answers
45 views

Equivalence of p-values from Jonckheere-Terpstra test and Kendall's tau

I'm using the Jonckheere-Terpstra test to confirm whether an interval variable, distributed across five levels of an ordinal variable, display a significant trend. I'm using the clinfun package, ...
2
votes
1answer
45 views

Big difference in p-value from permutation test

I have roughly 60 values of discrete temperature satisfaction votes ranging from 0 to 4. In the figure below I have grouped the votes depending on the daily number of hours the individual occupants ...
0
votes
0answers
17 views

Meta-analysis - generating null distributions of test statistics.

I'm having trouble obtaining p values for effect sizes from a meta-analysis because the distribution of t-statistics is not normal. The analysis involves eight studies, each containing measurements ...
0
votes
0answers
52 views

logistic regression with GLMM (and p-values)

It seems that I'm a little bit confused with the different ways to achieve its goals in the statistics’ world (and the R’s universe), so I’m asking your help about issues with which I'm struggling. ...
0
votes
0answers
51 views

Testing uniformity of data

I need to test if a vector of observed values are uniform distribution. Lets assume: This values are not a sample, but my entire universe. I have a dataset of 12000 observations, where most of the ...
1
vote
1answer
31 views

What does a large P value mean with regards to patients lost during follow-up?

What does "P=0.45" mean in this case? Complete 1-year follow-up data were obtained for 98.1% of the patients (11 were lost to follow-up in the angiography group and 8 were lost to follow- up ...
0
votes
1answer
33 views

Wilcoxon Rank Test or T-Test? [duplicate]

What data analysis is better? or fit for my data? I've used Likert scale for my questionnaire (1-5) and I have 387 respondents. Wilcoxon Rank Test or T-Test?