In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

learn more… | top users | synonyms (2)

1
vote
1answer
15 views

Finding “unloyal” customers with a Poisson distribution

we are trying to throw up warnings for when customers buying behavior might have changed. Our current indicator is the number of purchases in a given time period and we believe that this process ...
0
votes
1answer
10 views

p value in case of comparing several features

I have 2 populations A and B. I would like to test whether the difference in 100 features are significant between population A and population B. For each feature I compute the p value. Should I ...
0
votes
2answers
27 views

How to interpret the student's T-test

Let's say that we have 2 data samples $A$ and $B$, containing $N = 10000$ samples each, which are generated by two random variables $X_{A}$ and $X_B$. If I apply the student's T-test and apply a value ...
1
vote
2answers
52 views

Does a statistically significant correlation always give predictive power?

Suppose you're trying to predict anomalies. That is, consider the case where you have a data set that has a column called result. Suppose the data set has 365 rows and result has a value of 1 in ...
1
vote
0answers
19 views

How to calculate p-value for significance test of variables in linear regression

I am trying to find the significance of predictors while using different linear regression models (I am using Python scikit-learn). Scikit-learn does not provide the pvalues of predictors( at least i ...
0
votes
1answer
49 views

bootstrap p value in STATA

I have create a matrix with bootstrap estimation of the ATT. First I extract a sample using bsample command with cluster , then I estimate the propensity score ,and the ATT using the psmatch2 (with ...
0
votes
0answers
11 views

p-value for non normalized data [on hold]

I'm working on a project where I want to apply the output of a neural network in conjunction with a greedy (prediction) algorithm. The project is segmentation of 3D volumes. The greedy algorithm ...
0
votes
0answers
4 views

Distribution p-values for multiple correlation testing to nonzero value

I have data from psychological research with 44 variables (N=1748). I would like to compare all variables via correlations with FDR method to correct p-values significance level. When I tested to $r=...
6
votes
3answers
121 views

P-value word problem question

I'm having trouble understanding this question from my statistics class: I handed out a flyer about a rock show to 20 people who, at that time, did not plan on attending the show. My hypothesis is ...
0
votes
0answers
10 views

co-occurrence in hypergeometric model

My urn contains marbles that are either black, blue, red, or blue and red (if you're feeling poetic, they're ostraka in an amphora). I want to test whether my selected subset of ...
0
votes
0answers
18 views

Is there a statistical correction that works if the number of experiments depends on the result of the experiments?

Suppose that I am running an experiment that gives us some confidence $c$ about a hypothesis. If I intend to run the experiment $n$ times, I understand there are some ways of choosing a $c'$ so that ...
0
votes
1answer
18 views

Can univariate analysis find the p value, HZ ratio from the 100% alive group analysis

I try to compare two groups in univariate analysis but there are some problems because one group was alive only (no dead)
0
votes
0answers
5 views

Test statistic for comparison of family pairs vs. unrelated pairs generated by random sampling

I am working on an analysis to compare the difference between family pairs and unrelated pairs. Let us say we have 3 family pairs (A, B, C) and each pair have two members (1 and 2). To form unrelated ...
0
votes
2answers
26 views

How to calculate probability of success based on prior p-value

The FDA often requires a sponsor to conduct multiple clinical trials prior to approval. Given the following observations in a ph2 and ph3 trial, how would you go about predicting the probability of ...
1
vote
3answers
63 views

Number of false positives after Bonferroni or FDR

Let's say I am doing 100,000 tests. If the tests are independent, at an alpha of 0.05, we would expect (on average) 5000 of these tests to be false positives. The Bonferroni method would penalize ...
0
votes
0answers
20 views

Likelihood Ratio Test p-value for three parameters in r

I have a function with 3 unknown parameters (theta1, theta2, theta3), defined as: ...
1
vote
0answers
33 views

Contradictory p-value and confidence interval in permutation test

I am using the MICOM procedure to test for measurement invariance among groups in PLS SEM modelling [1]. This procedure consists of 3 steps, where in step 3 one tests for equality of composite means ...
1
vote
1answer
21 views

What is the right usage in Fisher test in R

So I have data in form (in real data I have more rows, of course so this is just a sample) ...
3
votes
1answer
24 views

Confidence interval for the p-value estimate when doing Monte Carlo testing

The second paragraph of the Monte Carlo testing section of the Wikipedia article on resampling statistics, the values of a confidence interval for a p-value from a MC sampling is given: After $N$ ...
2
votes
1answer
28 views

R: anova() vs. Anova() for test of categorical predictor from glmer or glm.nb object

In R, I'm wondering how the functions anova() (stats package) and Anova() (...
0
votes
1answer
25 views

What do you expect a 2-sample t-test for the same two samples to return?

I am using scipy in Python and the following return a nan value for whatever reason: ...
0
votes
0answers
23 views

Low coefficient of determination and low p-value

What should we think of a regression model that has a low coefficient of determination (e.g. $R^2 = 0.01$) and simultaneously a low p-value ($p<0.01$)? That in reality there is a non-random but ...
0
votes
2answers
32 views

Is it good practice to adjust for multiple comparisons when performing different multivariable regression models?

I have five different outcomes and the same group of independent variables which have been used for the five different multivariable Cox regression models. The indipendent variables I have used in the ...
1
vote
0answers
38 views

Is there an accepted definition for “P for trend” or “P-trend” [duplicate]

So is "P for trend" the same in principle as a common-or-garden P-value, but in a specific application i.e. odds ratios? And can P-value, P for trend, and P-trend be regarded as interchangeable ...
0
votes
1answer
44 views

Probability that the p-value is less than 0.05 if H0 is true?

Suppose that under $H_0$, a measurement $X$ is $N(0,\sigma^2 )$, and that under $H_1$, $X$ is $N(1,\sigma^2 )$ and that the prior probability $P(H_0) = P(H_1)$. With $\sigma$ = 1, what is the ...
1
vote
2answers
75 views

Rule of Thumb for Accepting the Null Hypothesis

Usually, hypothesis testing is performed with the goal to make a conclusions about the statistical significance of an effect, i.e. $H_0 \ \hat{=} \ \text{No Effect} $ vs. $H_1 \ \hat{=} \ \text{Effect}...
0
votes
0answers
22 views

unstandardised b coefficient, 95% CI and p-value interpretation in a linear regression

How could I interpret the table when: 1. the unstandardised coefficient has a value equal to 0? 2. either the lower or the upper bound of the 95% CI is 0 but the p-value is less than .05? I just run ...
6
votes
0answers
73 views

How do I statistically rephrase this question

I am analyzing a dataset containing observations from n number of attempts by players in a game. If I am building a regression model to predict the outcome of each attempt given 1 or more descriptors ...
-1
votes
1answer
37 views

how to predict significance (p-value?) of statistic outcome e.g. Brexit

I would like to know, what the significance of an outcome, e.g. that England gets out of the EU is, given that one has 6 samples of election districts (out of about 140), and in all 6, the outcome is ...
1
vote
0answers
27 views

Defining “p” value for glmm randomisation psuedo-p value

I am currently using a Bayesian approach to glmm (MCMCglmm in R) to estimate regression coefficients (see here where I ask whether randomisation of fixed effects is ...
1
vote
0answers
22 views

Adjusting p-values for multiple comparisons with correlated variables

I have three samples of size $N=10^6$ for the three responses $Y_1,Y_2,Y_3$ (or one sample of size $N$ for the random vector $\mathbf {Y}$). For each response the sample is i.i.d., even if the three ...
5
votes
2answers
72 views

t-statistic for linear regression

I know how to calculate t-statistics and p-values for linear regression, but I'm trying to understand a step in the derivation. I understand where Student's t-distribution comes from, namely I can ...
3
votes
1answer
86 views

How to ascribe and defend statistical significance in linear regression. P-value interpretation and alternatives

I am studying a treatment that degrades device quality and performed an independent sample test on two batches that had different initial quality. I analyzed the data with a two variable linear ...
1
vote
1answer
35 views

Sampling distribution of correlation coefficients and NHST

I often see people and libraries (e.g. here, here or here) reporting p-values anytime they measure correlation coefficients between two random variables. This always makes me wonder, what's exactly ...
1
vote
0answers
18 views

Using randomisation of fixed effects for a psuedo-p in mcmc glmm

I'm running generalized linear mixed models in R using the function MCMCglmm(), and with summary this generates p (pMCMC) values. However, pMCMC is, by the authors ...
0
votes
0answers
35 views

How to prove significantly high or low values?

I have the counts of genes in particular pathways from three organisms. How to statistically prove that few counts are significantly high or low as compared to another organism. My data looks like ...
0
votes
0answers
26 views

p-value for the McNemar's test

There's a question I wanted to ask with regards to the McNemar's test applied for case-control study. Suppose I have 2 models, I've fixed specificity and now I could construct the contingency table....
16
votes
2answers
596 views

High variance of the distribution of p-values (an argument in Taleb 2016)

I'm trying to understand the big picture claim made in Taleb, 2016, The Meta-Distribution of Standard P-Values. In it, Taleb makes the following argument for the unreliability of the p-value (as I ...
0
votes
0answers
13 views

Why var.test and levineTest produce very different p-values? [duplicate]

I run var.test in R using two samples sample1<-c(1,2,3) and sample2<-c(1, 10, 1000). I got a p value of 0.000006. When using Levene's test, I got a p value of 0.3689. SO is the variances equal ...
-5
votes
1answer
47 views

Why does a p-value of 2.4 indicate a very small probability that rejecting H0 is incorrect?

I am working with body temperatures in R, conducting a t-test. It is widely believed that the average body temperature for healthy adults is 98.6 degrees Fahrenheit. The body temperatures of $n = ...
8
votes
3answers
508 views

Hypothesis testing. Why center the sampling distribution on H0?

A p-value is the probability to obtain a statistic that is at least as extreme as the one observed in the sample data when assuming that the null-hypothesis ($H_0$) is true. Graphically this ...
0
votes
2answers
35 views

Can you derive Standard Deviations from a raw mean difference and p-value?

I have mean averages and a p-value, which I assume was of a t -test. This is the information provided to me: Treatment N =121 Control N=107 treatment mean = 356.700 treatment control= 357.560 ...
1
vote
1answer
36 views

Spearman's rho correlation coefficient and p-values

I am analysing data from questionnaires. I have 344 valid responses. When I run the spearman's rho correlation test to see if there's any correlation between students' home province and their ...
5
votes
1answer
71 views

How to correctly calculate the type I error in a two-step clinical trial when stopping after the first step?

I just finished reading the following article by Berger & Berry (1988) in which they explain how subjectivity enters statistical analyses. One of their examples concerns a clinical trial with ...
0
votes
0answers
15 views

Correspondence Analysis and p values

CA was originally designed as an exploratory method, and did not supply p values. So when mapping data one might have a table that looked at 3 texts of 500 words, each with counts for 10 words in ...
4
votes
4answers
434 views

A p-value greater than 0.05 means that my results are meaningless?

I just came across a correlation factor of 0.70 but p-value 0.054, does this mean that my correlation factor is completely meaningless and I should try to provide more samples?
0
votes
0answers
45 views

P for trend for quartiles in linear regression in SAS

I am using PROC GLM in SAS to run a linear regression model and have a continuous exposure variable that for a secondary analysis I have categorized into quartiles. I know I can use a contrast ...
1
vote
0answers
12 views

Random Walk: Right p-value method?

Based on a stochastic matrix/graph $G$, I want to conduct a series of random walks to get an idea which are frequent interaction partners for a starting node (not only connected by the first edge). I ...
0
votes
0answers
57 views

What statistical test to use when some participants receive multiple treatments?

I have N participants, some of them are on drugs A, B or C, exclusively (no overlapping). I know how to do the hypothesis test in this case. But what if I have a small subset, say n, of these ...