# Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

2,004 questions
Filter by
Sorted by
Tagged with
3 views

### Not understanding adjusted p values with Holm's method. Highest value in series divided by 1

I'm finding an issue while adjusting p-values with Holm's method I cannot understand. It's not a problem in my actual data, but I would appreciate any help to understand the issue. To show the ...
24 views

### How to properly calculate the exact p-value for binomial hypothesis testing (e.g. in Excel)?

I am confused with calculation of the p-value for binomial hypothesis testing when not using the normal approximation and its Z-statistic. My hypothesis testing is for the situation when H₀: π = π₀ ...
6 views

### How to handle multiple test correction between conventional p values and bootstrapped confidence intervals?

I have a correlation matrix and point estimates from mediation analyses all for a single research paper. I'm treating this paper as a single family of tests. The data is not normally distributed so ...
18 views

### Are the confidence intervals and p-value for this risk consistent?

I'm trying to calculate a risk ratio with the following 2x2 contingency table, where the rows are exposure groups and the columns are disease status. The first row of the matrix shows that 1/38 ...
11 views

### multiple regression with interrelated variables, significant only because of small sample size?

I am looking at the effect of different combinations of Height, Weight, and BMI on the age at which you sustain an injury. However, I only have 16 data points. R gives me an equation (similar to ...
16 views

### Correcting correlation coefficient significance (p-value) for dependent samples within Monte Carlo analysis

I would like to perform hypothesis testing on the significance of the correlation coefficient (Pearson's correlation coefficient) between variable $a$ and variable $b$. I do not have access to such ...
31 views

### K-S Test on estimated parameters? 22 Million datapoints

I know this question has been asked before, but I am having trouble with what I need to do in order to have valid results from KS test I have ~22 Million data points. Does this mean the p-values are ...
7 views

### Statistical Significance with Multivariate Logistic Regression and Feature Selection

Background: Given a dataset with output y (Bernoulli distribution) and features x_0, x_1, x_2, ..., x_n I want to test which ...
13 views

### Achieved significance level (ASL) vs p-value

Statement I have came across the achieved significance level recently. The rejection of the null hypothesis in a permutation test that I read about was based on the ASL. Question This has made me ...
32 views

### Large sample size creating significant variables

I'm currently working with a large sample size (around 400,000 observations) where I did a t-test and the p-value turned out to be less than 0.001 for all variables. This ofcourse is not something ...
7 views

### Sample Size Guidelines for KS-Test and AD-Test

so I've been looking into the Anderson Darling and Kologorov-Smirnov test because I have datasets with a large number of observations (~6000) and would like to fit distributions to them. They are ...
136 views

### Is post-selection inference a problem when robust tests are used?

It's pretty well acknowledged that error control via p-values fails when models are selected based on the data rather than decided on a priori. I've always viewed this as an issue of marginal vs ...
26 views

### How to interpret p-values in terms of correlation?

I have one datamatrix and a single column(let's say it fvector). I had find the spearman's correlation of each column from datamatrix with fvector i had setted the parameters as p <0.05 and i got ...
11 views

### Next step after building GLMM logistic regresion: LRT, ANOVA?

I am a biologist in the middle of their PhD journey and I'm constantly learning new things about statistics but there are some things that are difficult for me to understand. Until now I had dealt ...
19 views

### P values from lmer with lmerTest - why REML= TRUE?

I would like to obtain p-values from my model fit with lmer()from the lme4 package. It looks something like this: ...
16 views

### multiple logistic regression: removing not significant coefficients increases AIC score

I've built a logistic regression model using the glm function in R and ran a summary to look at the p-values of the coefficients ...
28 views

### how to compute p-value from Fisher's Z transformation?

I'm having a list of Pearson's correlation coefficients (r). Then, I performed the Fisher's Z transformation using the formula: z = 1/2 ln(1+r/1-r). Now, I want to compute corresponding p-values for ...
18 views

### Breusch Pagan and White test always 0

I'm working on a machine learning algorithm and trying to evaluate the model based on a guide online. No matter how I change the features of the model, the results of both tests are always 0. Is there ...
17 views

### How to determine which columns make p-value result in 0.0 in contingency table?

I have a contingency table with which I am estimating if some categories are seasonal or not. I use p-value and Cramers V for that but for this question only p-value is important. The p-value resulted ...
10 views

### why the p value of chi square test from factanal and principal functions in R opposite? [closed]

I am getting p-value for chi square statistic such that from factanal the null hypothesis is rejected and while for same data using principal it is such that fail to reject null hypothesis. Is that ...
22 views

### How do I analyze multiple treatments. For instance, what's the most rigorous way to statistically test 200 drugs?

I've been going down the rabbit-hole of multiple-testing, and I haven't figured it out quite yet. So let's take a simplified example. Let's say I have 200 drugs and I want to test their effectiveness. ...
35 views

### Manually calculating false positive risk (using Likelihood ratio and Bayes analysis)

The question is with reference to this paper: https://arxiv.org/pdf/1802.04888.pdf In the real life example on page 17-18, it is advised that the ...
14 views

### p-value for the coefficients of the logistic regression classifier

using Matlab, I have created two logistic regression classifiers: one with all the features (3) and one with only the first 2. I would expect to have the estimation of the p-value of the last variable ...
34 views

### (resolved) R: Clarification fisher.test (idem with chisq.test): Different p-values although same sample size, same proportion difference

I would appreciate so much if someone could help me to understand the following comparisons, and tell me if I am doing the tests correctly. I want to compare two districts and their respective ...
25 views

### Which reference could I cite for using confidence intervals instead of P-values when using robust linear mixed models (R package: robustlmm)?

I am using the R package robustlmm for some analysis due to its characteristics. I have read in posts like this one that with this type of models is better to use confidence intervals than $p$-values, ...
43 views

### Compare/Interpret t-statistics

I've a table as the following that contains the results of one sample t-test on four independent collections for the null hypothesis that the mean is 0: ...
38 views

### Moving away from p-values

I have seen some guides on how to craft your language when reporting statistical results (e.g., Smith (2020) https://doi.org/10.1007/s12237-019-00679-y) but does anyone know how to report multiple ...
22 views

### Large p-values have lower FDRs than small p-values

This a conceptual question. I ran 10 tests and sorted the p-values in an increasing manner. I calculate the FDR as p-value * (10 / rank) Then, in this case, the larger the p-value, the lower the ...
6 views

### hypothesis test to chose between two linear models assuming additive Gaussian error

I was trying to follow certain procedure that was done by some paper, and I don't fully understand how to do it. The paper describes as follow: The description is talking about 4 models and nested ...
13 views

### P value calculation for Anderson Darling normality

For p-value calculation I use (in c++) following equations I found on the internet ...
6 views

### Reporting significance values for relative risk (RR) and number needed to treat (NNT)

I have calculated the relative risk & number needed to treat using the log-binomial regression model. NNT is calculated as the reciprocal of the marginal difference. Does it make sense to report ...
21 views

### How do we explain together the results of paired t-tests and simple linear regression?

Description of my experiment: I have 2000 samples. For each sample I am measuring a metric Mi using four different methods A,B,C and D. These methods of measurement are completely independent. Ideally,...
25 views

### What does it means getting p-values equal to 1 and complete separation in logistic regression?

I was really confused if I should ask this here or in Stackoverflow, but I'll give a shot here. I ran a logistic regression using statsmodels library in Python. However, two things went wrong here (...
66 views

### Why does this formula produce $p_{2}$ probabilities for Mahalanobis distances?

At the IBM website it is written that The p1 probabilities are standard probabilities of an observation from a multivariate normal distribution being that far or further from the centroid, ...
13 views

### Deriving p-value from Fisher's z'

I am looking to derive a p-value to complete a hypothesis on the correlation between two independent variables. This is the data I have so far Pearson's r = 0.55 Fischer's z' = 0.618 How do I ...
35 views

### Parametric v Non-Parametric?

I've recently began doing some data analysis for a sample of around 5800 respondents who are categorised into four different groups in relation to different levels of psychological connection in a ...
104 views

### Why is bootstrapping p-values not just finding the null in the bootstrap samples?

I've been trying to look into the possibility of plucking a p-value for a slightly tricky case from the bootstrap distribution that I'm generating to construct confidence intervals. Everything I'm ...
28 views

### Why is the p value by average marginal effects different than the p value of the coefficients?

In R, when fitting a logit regression model, why is the p value for variable X different when finding its average marginal effect (AME) (using logitmfx) than when finding variable X's p value by its ...
20 views

### Why is the alpha value set by the authors of the paper and not me, the reader?

Lets say I read a paper, I think that the data collection method and the measurement practice will introduce noisy data. The sample is small and the underlying hypothesis is somewhat grandios. ...
29 views

### How to calculate a p-vale from the t-distribution for a two-tailed t-test?

The resulting $t$ statistic from a t-test follows a t-distribution with parameter $k =$ freedom-degrees. For us to compute the probability of $t$ to occur, we use the cumulative distribution function ...
25 views

### Noninferiority test statistics for Logistic Regression

I have two groups of samples: standard care, and alternative. I want to compare a specific outcome (e.g., mortality) of the two, but I want to control for different covariates. So I run a logistic ...
40 views

### converting a bootstrap confidence interval into a p-value

If a bootstrap confidence interval (CI) can be interpreted as a standard CI (e.g., the range of null hypothesis values that cannot be rejected) [also stated in this post]. Is it ok to derive a p-value ...
8 views

### Why does p-value have to be defined as 'at least as extreme' as the test statistic rather than just the probability that T= t? [duplicate]

Say I am using a t statistic, the (two-tailed-t test) p-value as i understand is given as pr(t$\geq$|t| |null is true), why cant it be that pr(t=t|null) or at least some small relative frequency in an ...
14 views

### P-values and confidence intervals for Inverse Gaussian GLMM?

I want to test the effects of different variables on Reaction Times data. Following the recommendations of Lo & Andrews (2015) I compared the AIC/BIC of three GLMM with a Gaussian, a Gamma and an ...
25 views

### Which mathematical test do I need to use?

I have a dataset showing the yearly stock returns for two investment styles (growth and value investing) as well as a column showing the difference between the returns for each year. I have attached a ...
17 views

### Significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...
15 views

### Interpret repeated measures ANOVA results in R

I developed many model on groups of data I applied repeated measures anova to check the significant difference. Then I applied pairwise.t.test using Bonferroni correction. I got this result from the ...
31 views

### Help for interpretation : significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...