In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

learn more… | top users | synonyms (2)

0
votes
0answers
21 views

What to do if a categorical variable has some components that are not statistically significant?

I'm trying to construct a univariate prediction model using logistic regression in order to predict credit default likelihood from overdue level in telecommunication companies: ...
0
votes
0answers
34 views

Regression ceoff. p-value as a 'summary measure' of 'indication of relative importance'

I have a paper in front of me (peer reviewed, etc.), which states "comparison of the P values for [the covariates] gives a good indication of the relative importance of each [covariate] for ...
1
vote
1answer
17 views

large sample and significance

I have a very large sample (790 million rows), and in this case, even small difference are determined to be significant, p <0.0001. The 95% C.I.s are very sharp. In this case, these tests (t-test, ...
1
vote
0answers
21 views

Why do p-values have uniform distribution for normally distributed data [duplicate]

I generated some random data using rnorm() in R made a matrix X of 10,000 rows and 1,000 rows. The I generated 1000 zeros and ones using runif(). I stored the rows corresponding to 0 in Y in one ...
0
votes
0answers
24 views

Odds ratio confidence interval and p-value do not agree [closed]

I calculated an odds ratio of 0.125, p-value (alpha 0.05) = 0.0378. However, the 95% CI crosses 1. (0.0143, 1.0635). I am wondering how I can reconcile this difference. Ideally I would like to report ...
0
votes
0answers
8 views

Dunnett T3 test interpretation

i have result from a Dunnett T3 test using package DTK in R. The result do not give me any p-value but only "Diff", "lower CI" and "upper CI". i can understand the difference between groups as the ...
6
votes
1answer
46 views

Why does Bayesian p-value involve the parameters in addition to the data?

On page 146 of Gelman's Bayesian Data Analysis, Gelman discusses Bayesian p-value as a way to check the fit of the model. The idea is to compare the observed data ($y$) with data that could have been ...
0
votes
0answers
19 views

calculate p-value for AFT survival model

I am using Spark ml library to do some survival analysis. Here is the documentation. After training an AFT survival model, I cannot get the p-value directly as in R. What is available for the model ...
8
votes
5answers
159 views

How to test an interaction effect with a non-parametric test (e.g. a permutation test)?

I have two categorical / nominal variables. Each of them can take only two distinct values (so, I have 4 combinations in total). Each combination of values comes with a set of numerical values. So, I ...
1
vote
0answers
19 views

Prevalence of the “vote counting” method in literature reviews

In my youth I can remember reading some papers (in psychology most likely) which as part of their literature review included a table of past results with no details provided other than whether the ...
0
votes
0answers
8 views

two sided ks test loop, get p.value [migrated]

I have a column of data from which I am taking randomised sub samples of 50%. I'm running a two sided ks test to compare the distribution of 50% of the data against 100% of the data to see if the ...
0
votes
0answers
36 views

p-value interpretation of Jonckheere Terpstra:one-sided or two-sided

I am using the Jonckheere-Terpstra test in order to validate the pooled t-test results. Basically, I like the JT test because of its non-parametric nature. Basically, I am now trying to understand ...
0
votes
0answers
29 views

Intuitions from standard deviation to p-value?

A cake is specified by a 3-vector $x=(x_1,x_2,x_3)$, where $x_i$ is the percentage of the $i$-th ingredient, and therefore $\sum_ix_i=1$. I have a price list $y$ for different $x$'s: ...
1
vote
1answer
62 views

Hypothesis testing with no sample or standard deviation, is this is a trick question?

The question I was asked is as follows: A website received 6802 page views last week. Does this provide evidence at a 1% significance level against the claim that the website receives on average 1000 ...
1
vote
2answers
225 views

Do p-values assume an unlikely event has already happened?

According to this thread What is the meaning of p values and t values in statistical tests? a p-value essentially lets me compare the probability of tail-end (unlikely) events occurring and judging ...
0
votes
1answer
13 views

Mixed-effect models for repeated measures - problems with the “Significance level”

I am trying to fit a linear mixed-effect model to my dataset to see the relationship between a self-reported questionnaire and some physiological data. I've created a first model including all the ...
0
votes
0answers
15 views

Why does a low p-value mean there is sufficient evidence to reject the null hypothesis? [duplicate]

As I understand it, a p-value is the prob of getting something as far or farther out in the tails of a normal distribution than the observed t value. source In other words, $$\Bbb P(\text{value ...
0
votes
1answer
25 views

Combine multiple independent permutation tests

I want to find out whether a neural measure measured at 12 time-points is correlated with a behavioral measure measured on the same time-points. I have data from 20 subjects. I'm hoping to perform a ...
0
votes
0answers
11 views

Can I extrapolate a mean difference from the t-statistic?

I have the t-statistic, df, p and cohen's d for the one-tailed independent samples t-tests on the difference between a parameter for cases and controls. I would like to obtain a mean difference and ...
0
votes
0answers
20 views

Can I obtain a mean difference from fold change data, number of participants and p values?

I have gene expression in fold change for cases vs controls. Therefore, below I have 4 times the gene expression of gene x in the cases compared with the controls. I have the number of participants ...
1
vote
0answers
27 views

Interpreting pvalue in the light of known power

I'm puzzled with the interpretation of a pvalue in the light of known power Let's say I study the association between a dichotomic variable (a treatment) and a quantitative one, which is my primary ...
1
vote
0answers
24 views

Calculating p-value based on Chi-Square vs Z-Test incl Standard Error

I have been testing the significance levels for my various a/b tests. Degree of freedom is 1 so there's not a lot of complications here. So far I've been calculating my results using Chi-Squared ...
0
votes
1answer
32 views

Does the p value for logistic regression depends on odds ratio or logit?

The p value in a logistic regression is given for the B estimates. But what if I'm reporting odds ratio instead of logit. Do the same p values apply?
7
votes
2answers
102 views

Robustness of correlation test to non-normality

I'm trying to reconcile two seemingly opposite statements about robustness to non-normality of the Pearson's correlation test statistic (where the null means "no correlation"). This CV answer says: ...
0
votes
0answers
17 views

Best approach for calculating confidence score (p-value) - for Bayes models

This is a generic question on the best approach for giving a p-value to a "most likely" solution given by a graphical Bayes model. I am building a graphical model in Stan, for finding the most likely ...
1
vote
1answer
41 views

How to say “the statistical significance of our result is stronger”?

I'm doing a Bonferroni correction, after which our result is still significant at the 10% level. Since my multiple outcomes are highly correlated, for which the Bonferroni is too conservative, I want ...
0
votes
1answer
43 views

Contradictory results of the ADF test in R [closed]

I used the urca pacakge in R to run the ADF test and got the following: It seems to me the results are quite contradictory: Using the statistic, we fail to reject the null hypothesis; However, ...
0
votes
0answers
16 views

Interpreting Tukey HSD p-value After ANOVA + FDR correction

I'm currently analyzing a dataset where I have gene expression pathway activation scores across three groups: two different treatments and one control (so, basically a matrix of the form pathways x ...
0
votes
1answer
47 views

P-value & Confidence Interval…I choose you CI!

I'm struggling with an interpretation of p-value and confidence interval. They are both in agreement, but yet, can't be used together to make one statement about population proportion. So...am I ...
0
votes
0answers
7 views

Unsure of how to test a mixture model coefficient and standard error.

This is my first question on Stack Exchange so please excuse any faux pas! I've been working with a mixture model from the paper "Efficient estimation for an accelerated failure time model with a ...
0
votes
0answers
21 views

How to adjust p-value to reject null hypothesis from sample size in Mann Whitney U test?

I am trying to test if the shape and median of two samples are significantly different using Mann Whitney U test (or wilcox.test in R). The two samples (A and B), ...
0
votes
0answers
26 views

Machine Learning Algorithms For Poor Quality Statistics

I'm looking for references on any work done in the area of machine learning for automated detection of poor quality papers that use statistics (for example p-value hacking, misunderstanding confidence ...
0
votes
1answer
64 views

Conclusion from Hypothesis Testing and CI

Can you draw a conclusion from combining results of hypothesis testing and confidence interval? For example, here is a fictitious hypothesis test for 1-proportion: Ho = p ≥ 90% Ha = p < 90% ...
1
vote
1answer
67 views

does p-value distribution indicate the suitability of the statistical test

This post suggests that p-value follows uniform distribution in case of point null hypothesis and continuous data. In my project, I have millions of p-values from actual data. I don't want to get ...
0
votes
0answers
27 views

Hypothesis Testing Fictitious Study Help

I'm doing a self-study on hypothesis testing and have created a fictitious study to facilitate my learning. I just need validation, from you experts, that my thought process on the concepts and ...
17
votes
0answers
199 views

I'm more likely to read “interesting” studies, so do all p-values I read suffer a “readership bias”?

Do I bias p-values simply by reading them — or, more precisely, by having chosen to read the study that contains them? I generally take most p-values with a sceptical pinch of salt. Sometimes ...
0
votes
0answers
30 views

Effect of Federal Funds Rate on SP500

I have been trying to examine the effect of the Federal Funds Rate on the S&P 500 using regression analysis. Obviously via the graph its easy to see this is a hard relationship to define over a ...
0
votes
0answers
33 views

Hypothesis testing: t-test and p-value conflict

I am doing a one-tailed hypothesis test on the exper coefficient. My null is the coefficient is not different from zero and my alternative is that the coefficient is greater than 0. I'm using a 5% ...
0
votes
0answers
10 views

Insignificant alone but significant when inclued in multi variate regression [duplicate]

I have a target variable y, two explanatory variables x1 and x2. x1, x2 and y all have zero mean. x1 and x2 has low correlation. y = b1 * x1 + e gives R^2 = 0.6, p-value of b1 = 0.000, out of sample ...
0
votes
0answers
17 views

For a standard normal distribution, are p-values calculated using a $\Phi(-Z)$ or $\Phi(Z)$?

Let $\bar{X}$ denote the value of the sample average for a given set of iid random variables $\lbrace X_i\rbrace_{i=1}^n$. Assume each $X_i \sim \mathcal{N}(0,1)$. According to my econometrics ...
1
vote
1answer
52 views

What does it mean if test was rejected less than in 5% cases?

I was reading a paper and authors performed multiple tests for Poisson distribution on single vectors. Large dataset of $n$ quite long vectors was used. The test was rejected only $0.02 \cdot n$ times ...
1
vote
0answers
28 views

Interpretation of Mann-Kendall statistics

I am doing a trend analysis for vegetation dynamics using Mann-Kendall test for trend detection and Sen's slope for computing the magnitude of the trend. There are 6 sites inside a conservation area ...
0
votes
0answers
41 views

pairwise.t.test in R returning no value

I am a noob in R, and have been breaking my head for the past three hours over the pairwise.t.test in R. My data frame is structured as given below. My data frame has the age of flight of bees and the ...
0
votes
0answers
13 views

Seemingly random results from kstest2 function in MATLAB

I'm trying to understand the kstest2 function in MATLAB so I can use it to check if output samples from two models are from the same distribution. The first thing I did was run a test to make sure ...
0
votes
0answers
38 views

Beta regression model - how to test the model to obtain p values?

I have coral cover data (in percent coverage, thus it is between 0 and 100). I am using the betareg package in R to fit a multiple regression model, where Coral ...
0
votes
0answers
36 views

What do the four coloured areas of this p-value distribution actually represent? [duplicate]

The chart below shows a distribution of p-values (deep blue and orange areas) given by a data sample that deviates from the null hypothesis, while the grey and light orange areas represent the ...
1
vote
1answer
74 views

What is the link (discrepancy?) between these PDF/CDF and p-value distributions?

I have created a mixed distribution model comprising 80% $H_0$ plus 20% $H_1$ to illustrate the link between the expected proportions of true and false positives and negatives in the PDF, CDF and ...
1
vote
1answer
30 views

Understanding the statistical significance calculated in an experiment

I am trying to understand analysis in a paper which talks about statistical significance of a certain stimulus. I have used 'VALUE' in place of the specific variable that is being observed, and ...
1
vote
1answer
73 views

Is it fair to use FDR when the p-value distribution is not uniform under null hypothesis?

One post says that we should check the distribution of p-value before apply FDR correction. If the p-value distribution doesn't behavior well(e.g. U shape, not uniformly distributed at the tail toward ...
0
votes
0answers
16 views

Degrees of freedom and p-value calculation using contingency coefficient value?

1) I evaluated Chi-Sqr test for 18 rows and 5 columns table. Total Chi-Square value is 128.5 and calculated Contingency coefficient value CC =0.048. Now how to calculate P-value and also how to ...