In hypothesis testing, the $p$-value refers to the probability of seeing a result as extreme (or more so) than the one observed, given that the null hypothesis is true. When the $p$-value is small, then there is good evidence that the null hypothesis is false, since it would make the observed data ...

learn more… | top users | synonyms (2)

3
votes
2answers
26 views

Association between two proportions using p-value

I have two proportions and I need to see if they are associated somehow. The null hypothesis is that they are equal. I'm using R. So I get the pooled proportion: ...
20
votes
4answers
1k views

Is p-value essentially useless and dangerous to use?

First are some background information. This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that [Bayesian statistics] is proving ...
0
votes
0answers
16 views

minitab p-values of interactions

I am having some trouble in Minitab. I constructed a L16 Taguchi orthogonal array for 5 factors which all have 2 levels by using Minitab, and I need to check if all the interactions are significant or ...
4
votes
2answers
143 views

Why is the p-value distribution not uniform?

While learning on the q-value http://www.totallab.com/products/samespots/support/faq/pq-values.aspx I saw that, under the null-hypothesis, the distribution of the p-value is expected to be uniform. ...
2
votes
1answer
41 views

One model performs better than the other. How to measure if it is statistically significant?

So, let's say that I train two models on the same dataset. I run the experiment once and I get the following results: Using a Neural Network I get an AUC ROC of 0.941. Using Random Forest I get an ...
5
votes
2answers
44 views

How to get p-values or confidence intervals for pearson correlation coefficient when the sample is small and potentially non-Gaussian?

The p-values provided by the built-in Matlab and Pylab Pearson correlation functions are stated to be inaccurate for small sample sizes, or when the samples are not normally distributed. The python ...
2
votes
1answer
40 views

Do different units of explanatory variables affect p-values?

In multiple linear regression, does including variables with different metering units and thus values very different in dimension/size affect statistical significance/ p-values? In my model, a ...
0
votes
2answers
54 views

Best classical statistics books

This question has been asked many times before but I didnt find great references - so here's another try :) I am looking for basic statistics book that would be of the level of advanced undergrad. ...
0
votes
4answers
57 views

Interpreting high p value and low correlation value

I am trying to run regression on financial data in R. I am new to regression analysis so I am finding it to difficult to interpret certain scenarios. I have the code as follows: ...
0
votes
2answers
44 views

p value vs prediction error

In a lot of fields (like medicine) to check if a variable is related to an output is controlled if the p-value of that variable in a regression model is significant. For example: ...
0
votes
0answers
32 views

Combining p-values from multiple imputed data sets

I have created a set of imputed data sets using 'mice'. The data is longitudinal in nature, with both a longitudinal outcome and survival outcome of interest, and time-varying predictors. I conducted ...
2
votes
0answers
20 views

Alternatives to Fisher's method?

I'm using the Mann-Whitney U test to compare scores from a foreground (n ranging between 5-500) to a large background (up to 2000). Since my foreground is quite small I've been taking smaller random ...
2
votes
1answer
44 views

Manually calculating p-value for t-test: How to avoid values bigger $1$?

These two methods for calculating the p-value should be equivalent: t.test(rats.drug,mu=1.2)$p.value 2*pt((mean(rats.drug)-1.2)*sqrt(n)/sd(rats.drug),df=n-1) The ...
2
votes
1answer
160 views

Why is the difference between these two t-tests so big?

I am using a t-test to find out whether the mean of a sample ($1.05$) with a sample size of $100$ is significantly different from a given mean ($1.2$). The standard deviation should be $0.5$: ...
4
votes
2answers
653 views

What is this equation (pictured) called?

Could someone provide some Google-able words for this equation? $$Q_{\chi^2,d} = \left[2^{d/2}\Gamma\left(\frac{d}{2}\right)\right]^{-1}\int_{\chi^2}^\infty (t)^{\frac{d}{2}-1}e^{-\frac{t}{2}}dt$$ ...
1
vote
1answer
45 views

Hypothesis testing with group of stocks

I'm in a situation where I need to compute a p-value relating to the hypothesis that the mean daily return from a strategy applied to a group of stocks is different from zero. The usual way to do this ...
0
votes
0answers
10 views

P-values for random effects when using REML [duplicate]

I'm using JMP to fit a model for an unbalanced split-plot design. Because it's unbalanced, I'm using REML rather than EMS. However, I would like to get test statistics/p-values for some of the random ...
1
vote
0answers
10 views

Multiple Ancova's and adjustments to p-values

I am running about 10 Ancova's and am unsure about whether I adjust the overall p-value and then also the posthoc analyses p values. If the categorical independent factor has 3 or more levels. do I ...
2
votes
0answers
34 views

ANOVA P-value adjustments for semi-continuous data

In tests such as the Bonferroni post-hoc p-value adjustment test, convention is to correct the data with a constant, equivalent to the length (number) of pairwise comparisons that are being performed. ...
0
votes
0answers
8 views

p.adjust: concerns for FDR-adjustment where n not equal to length(p)

Specific Question: When using p.adjust to convert p-values to FDR-adjusted q-values, what are the conditions that need to be met to specify an 'n' that is not equal to length(p), where p is the vector ...
0
votes
0answers
38 views

Grasping the concept of P-value [duplicate]

I am having a hard time grasping the concept of the P-value.. I've read some of the posts in here and on internet, but one thing seem to be missing for me. Usually we look at the probability of ...
2
votes
4answers
221 views

Why do I get this p-value doing the Jarque-Bera test in R?

Doing a Jarque Bera test in R I get this result: jarque.bera.test(rnorm(85)) data: rnorm(85) X-squared = 1.259, df = 2, p-value = 0.5329 Does it mean that the ...
0
votes
0answers
24 views

Are combined P values derived from FDRs also corrected for multiple testing?

Background The problem I am going to describe has to do with analysis of microarray data, and measurements are made at probes and p.values are returned for differences between groups at probes. The ...
3
votes
3answers
133 views

glm in R - which pvalue represents the goodness of fit of entire model?

I am running glms in R (generalised linear models). I thought I knew pvalues - until I saw that calling up a summary for a glm does not give you an overriding pvalue representative of the model as a ...
0
votes
1answer
56 views

glm in R - is the pvalue for Intercept important? Which pvalue represents the goodness of fit of entire model?

I am running glms in R (generalised linear models). I thought I knew pvalues - until I saw that calling up a summary for a glm does not give you an overriding pvalue representative of the model as a ...
0
votes
1answer
37 views

Resources to understand why dependence is a problem [duplicate]

For many statistical procedures, it seems the observations must be independent. For example the observations within each group in a two-sample t-test must be independent for the standard error/P-value ...
4
votes
1answer
46 views

Understanding multiple KS tests

I have read these two questions Why are p-values uniformly distributed under the null hypothesis? and Understanding scipy Kolmogorov-Smirnov test And this inspired me the following experiment. I ...
1
vote
1answer
21 views

Combining multiple p-values

Is there any statistically relevant way to combine multiple p-values measuring different hypotheses but that are all trying to measure the same thing "in spirit"? Eg, if I wanted to measure some ...
8
votes
5answers
149 views

What is the relationship between $p$ values and Type I errors [duplicate]

In hypothesis testing we set an accepted level of Type I error probability $\alpha$ and observe whether a sample statistic is equally likely or less likely to be observed if the null hypothesis was ...
0
votes
1answer
49 views

Interpreting high P-Value in ordinal rank correlation measures

I'm currently evaluating a machine-learned ranking task. Two (among other) performance measures are Spearman's Rho and Kendall's Tau (rank or concordance correlation). I'm basically comparing a ...
2
votes
0answers
29 views

How is plot of p-values distribution related to calculation of the false discovery rate?

I'm running t-tests for all combinations of pairwise comparisons of average length of stay for patients among 30 hospitals. My first approach was to use a Bonferroni corrected significance level of ...
2
votes
0answers
72 views

Converting Chi Square Adjusted Residuals to p values

I am running a chi squared analysis in SPSS on a 5 x 2 contingency table. The overall chi square p value was < 0.0005, so I ran post hoc tests (adjusted standardized residuals on the individual ...
3
votes
0answers
63 views

How to calculate p-value using an algorithm on R assuming that distribution is unknown

For a given sample x, where x is distributed with a normal distribution mean known but variance unknown. I am testing the hypothesis that variance is equal to one or greater than 1. The question is to ...
3
votes
1answer
53 views

p-value correction for multiple t-tests?

My dataset consists of $n$ genes, each of them described by a vector of expression values, $5$ for "healthy" individuals, and $5$ for "unhealthy" individuals. I am going to run $n$ t-tests (one for ...
0
votes
0answers
47 views

understanding of p-value in multiple linear regression

Regarding the p-value of multiple linear regression analysis, the introduction from Minitab's website is shown below. The p-value for each term tests the null hypothesis that the coefficient is ...
3
votes
0answers
21 views

Determining statistical significance in difference between two CRF models

I have trained two Conditional Random Field (CRF) models Ma and Mb in CRF++, the first using a simple set of features, the ...
2
votes
0answers
27 views

Stouffer's method in log space

I'm combining p-values using Stouffer's method in R using the code from the linked Wikipedia page. Since some p-values can be close to 0 the combined p-value becomes 0 as a result of underflow (I ...
0
votes
1answer
39 views

How to obtain p-values for a given Kendall's Tau value (not calculated from data)

I want to obtain p-values for Parker's Tau-U (Parker et al., 2011). which is a measure based on Kendall's Tau. Tau-U is related to Kendall's Tau in such a way that it's possible to translate a Tau-U ...
2
votes
0answers
43 views

Running sequential tests as data is collected - practical advice for researchers

I've observed that many experimental labs in my field don't do a power analysis before they start collecting data. In practice, it seems that many researchers collect data and periodically run ...
2
votes
1answer
45 views

How can I calculate p-values in R?

I am trying to construct a hypothesis test for the variance of two populations. My test statistic is $F=$${\sigma_b^2}$/${\sigma_A^2}$. I want to implement this as a function in R so I computed: ...
1
vote
0answers
24 views

Error propagation and p value

Say I have parameters $a(x$) and $b(x)$, with $N_a(x)$ and $N_b(x)$ experimental measurement counts, respectively. $x$ is an independent variable ($x_1, x_2, ...$). For each $a$ and $b$, I can ...
0
votes
1answer
132 views

Why is P value interpreted under null hypothesis? [duplicate]

P value is explained very nicely in the link . But still i have not understood Why is P value interpreted under null hypothesis?
0
votes
1answer
37 views

Permutation test and p-value

I construct a permutation test in order to see If two samples come from the same distribution or not. I have two vectors $x,y$ that hold values of sampled values from two populations and the test ...
0
votes
1answer
34 views

Excel: Computing a pvalue [closed]

An event has an expected probability of .4. After 100 trials, the event occurred 30 times. How can the pvalue be computed in Excel?
3
votes
1answer
95 views

sample size too large? [duplicate]

I always thought larger sample sizes were better. Then I read something somewhere about how when sample sizes are larger, it's easier to find significant p-values when they're not really there (i.e., ...
10
votes
2answers
399 views

p-value subtlety: greater-equal vs. greater

As I am reading Wassermann's book All of Statistics, I notice a fine subtlety in the definition of p-values, which I cannot make sense of. Informally, the Wassermann defines the p-value as [..] ...
1
vote
1answer
74 views

Distribution of p-values in this thought experiment?

I'm trying to check whether my clustering was informative above and beyond random clustering. This is my thought experiment to do it, can someone help? Suppose I have a large number, $N$, groups. ...
1
vote
1answer
67 views

Bootstrap glm and extract pvalue

I am running a glm model using bootstrap, I can extract the coefficient mean and the confidence intervals for all the factors in my model. But how can I get the pvalue from there? Model: ...
0
votes
0answers
16 views

Compute effect size based on simulation using different lists of different length

For my research purpose I wrote a small simulation to compute an empirial p-value based on an input gene list and a reference gene list. The idea is to check for each gene in the input gene list, the ...
1
vote
0answers
41 views

Tukey HSD, insignificant p-value when the confidence interval does not contain 0?

I am using TukeyHSD to do multiple comparisons. I am seeing strange results for some of the pairs: the adjusted p-value is greater than the specified significance level but the corresponding ...