Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

Filter by
Sorted by
Tagged with
0
votes
1answer
3 views

Not understanding adjusted p values with Holm's method. Highest value in series divided by 1

I'm finding an issue while adjusting p-values with Holm's method I cannot understand. It's not a problem in my actual data, but I would appreciate any help to understand the issue. To show the ...
0
votes
0answers
24 views

How to properly calculate the exact p-value for binomial hypothesis testing (e.g. in Excel)?

I am confused with calculation of the p-value for binomial hypothesis testing when not using the normal approximation and its Z-statistic. My hypothesis testing is for the situation when H₀: π = π₀ ...
0
votes
0answers
6 views

How to handle multiple test correction between conventional p values and bootstrapped confidence intervals?

I have a correlation matrix and point estimates from mediation analyses all for a single research paper. I'm treating this paper as a single family of tests. The data is not normally distributed so ...
-1
votes
1answer
18 views

Are the confidence intervals and p-value for this risk consistent?

I'm trying to calculate a risk ratio with the following 2x2 contingency table, where the rows are exposure groups and the columns are disease status. The first row of the matrix shows that 1/38 ...
0
votes
0answers
11 views

multiple regression with interrelated variables, significant only because of small sample size?

I am looking at the effect of different combinations of Height, Weight, and BMI on the age at which you sustain an injury. However, I only have 16 data points. R gives me an equation (similar to ...
1
vote
0answers
16 views

Correcting correlation coefficient significance (p-value) for dependent samples within Monte Carlo analysis

I would like to perform hypothesis testing on the significance of the correlation coefficient (Pearson's correlation coefficient) between variable $a$ and variable $b$. I do not have access to such ...
1
vote
1answer
31 views

K-S Test on estimated parameters? 22 Million datapoints

I know this question has been asked before, but I am having trouble with what I need to do in order to have valid results from KS test I have ~22 Million data points. Does this mean the p-values are ...
0
votes
0answers
7 views

Statistical Significance with Multivariate Logistic Regression and Feature Selection

Background: Given a dataset with output y (Bernoulli distribution) and features x_0, x_1, x_2, ..., x_n I want to test which ...
0
votes
0answers
13 views

Achieved significance level (ASL) vs p-value

Statement I have came across the achieved significance level recently. The rejection of the null hypothesis in a permutation test that I read about was based on the ASL. Question This has made me ...
1
vote
2answers
32 views

Large sample size creating significant variables

I'm currently working with a large sample size (around 400,000 observations) where I did a t-test and the p-value turned out to be less than 0.001 for all variables. This ofcourse is not something ...
0
votes
0answers
7 views

Sample Size Guidelines for KS-Test and AD-Test

so I've been looking into the Anderson Darling and Kologorov-Smirnov test because I have datasets with a large number of observations (~6000) and would like to fit distributions to them. They are ...
4
votes
2answers
136 views

Is post-selection inference a problem when robust tests are used?

It's pretty well acknowledged that error control via p-values fails when models are selected based on the data rather than decided on a priori. I've always viewed this as an issue of marginal vs ...
0
votes
1answer
26 views

How to interpret p-values in terms of correlation?

I have one datamatrix and a single column(let's say it fvector). I had find the spearman's correlation of each column from datamatrix with fvector i had setted the parameters as p <0.05 and i got ...
0
votes
0answers
11 views

Next step after building GLMM logistic regresion: LRT, ANOVA?

I am a biologist in the middle of their PhD journey and I'm constantly learning new things about statistics but there are some things that are difficult for me to understand. Until now I had dealt ...
0
votes
0answers
19 views

P values from lmer with lmerTest - why REML= TRUE?

I would like to obtain p-values from my model fit with lmer()from the lme4 package. It looks something like this: ...
0
votes
0answers
16 views

multiple logistic regression: removing not significant coefficients increases AIC score

I've built a logistic regression model using the glm function in R and ran a summary to look at the p-values of the coefficients ...
0
votes
1answer
28 views

how to compute p-value from Fisher's Z transformation?

I'm having a list of Pearson's correlation coefficients (r). Then, I performed the Fisher's Z transformation using the formula: z = 1/2 ln(1+r/1-r). Now, I want to compute corresponding p-values for ...
0
votes
1answer
18 views

Breusch Pagan and White test always 0

I'm working on a machine learning algorithm and trying to evaluate the model based on a guide online. No matter how I change the features of the model, the results of both tests are always 0. Is there ...
0
votes
0answers
17 views

How to determine which columns make p-value result in 0.0 in contingency table?

I have a contingency table with which I am estimating if some categories are seasonal or not. I use p-value and Cramers V for that but for this question only p-value is important. The p-value resulted ...
0
votes
0answers
10 views

why the p value of chi square test from factanal and principal functions in R opposite? [closed]

I am getting p-value for chi square statistic such that from factanal the null hypothesis is rejected and while for same data using principal it is such that fail to reject null hypothesis. Is that ...
1
vote
0answers
22 views

How do I analyze multiple treatments. For instance, what's the most rigorous way to statistically test 200 drugs?

I've been going down the rabbit-hole of multiple-testing, and I haven't figured it out quite yet. So let's take a simplified example. Let's say I have 200 drugs and I want to test their effectiveness. ...
0
votes
1answer
35 views

Manually calculating `false positive risk` (using Likelihood ratio and Bayes analysis)

The question is with reference to this paper: https://arxiv.org/pdf/1802.04888.pdf In the real life example on page 17-18, it is advised that the ...
0
votes
0answers
14 views

p-value for the coefficients of the logistic regression classifier

using Matlab, I have created two logistic regression classifiers: one with all the features (3) and one with only the first 2. I would expect to have the estimation of the p-value of the last variable ...
0
votes
2answers
34 views

(resolved) R: Clarification fisher.test (idem with chisq.test): Different p-values although same sample size, same proportion difference

I would appreciate so much if someone could help me to understand the following comparisons, and tell me if I am doing the tests correctly. I want to compare two districts and their respective ...
0
votes
1answer
25 views

Which reference could I cite for using confidence intervals instead of P-values when using robust linear mixed models (R package: robustlmm)?

I am using the R package robustlmm for some analysis due to its characteristics. I have read in posts like this one that with this type of models is better to use confidence intervals than $p$-values, ...
2
votes
1answer
43 views

Compare/Interpret t-statistics

I've a table as the following that contains the results of one sample t-test on four independent collections for the null hypothesis that the mean is 0: ...
1
vote
1answer
38 views

Moving away from p-values

I have seen some guides on how to craft your language when reporting statistical results (e.g., Smith (2020) https://doi.org/10.1007/s12237-019-00679-y) but does anyone know how to report multiple ...
2
votes
1answer
22 views

Large p-values have lower FDRs than small p-values

This a conceptual question. I ran 10 tests and sorted the p-values in an increasing manner. I calculate the FDR as p-value * (10 / rank) Then, in this case, the larger the p-value, the lower the ...
0
votes
0answers
6 views

hypothesis test to chose between two linear models assuming additive Gaussian error

I was trying to follow certain procedure that was done by some paper, and I don't fully understand how to do it. The paper describes as follow: The description is talking about 4 models and nested ...
1
vote
0answers
13 views

P value calculation for Anderson Darling normality

For p-value calculation I use (in c++) following equations I found on the internet ...
1
vote
0answers
6 views

Reporting significance values for relative risk (RR) and number needed to treat (NNT)

I have calculated the relative risk & number needed to treat using the log-binomial regression model. NNT is calculated as the reciprocal of the marginal difference. Does it make sense to report ...
0
votes
0answers
21 views

How do we explain together the results of paired t-tests and simple linear regression?

Description of my experiment: I have 2000 samples. For each sample I am measuring a metric Mi using four different methods A,B,C and D. These methods of measurement are completely independent. Ideally,...
2
votes
1answer
25 views

What does it means getting p-values equal to 1 and complete separation in logistic regression?

I was really confused if I should ask this here or in Stackoverflow, but I'll give a shot here. I ran a logistic regression using statsmodels library in Python. However, two things went wrong here (...
2
votes
1answer
66 views

Why does this formula produce $p_{2}$ probabilities for Mahalanobis distances?

At the IBM website it is written that The p1 probabilities are standard probabilities of an observation from a multivariate normal distribution being that far or further from the centroid, ...
0
votes
0answers
13 views

Deriving p-value from Fisher's z'

I am looking to derive a p-value to complete a hypothesis on the correlation between two independent variables. This is the data I have so far Pearson's r = 0.55 Fischer's z' = 0.618 How do I ...
0
votes
0answers
35 views

Parametric v Non-Parametric?

I've recently began doing some data analysis for a sample of around 5800 respondents who are categorised into four different groups in relation to different levels of psychological connection in a ...
6
votes
1answer
104 views

Why is bootstrapping p-values not just finding the null in the bootstrap samples?

I've been trying to look into the possibility of plucking a p-value for a slightly tricky case from the bootstrap distribution that I'm generating to construct confidence intervals. Everything I'm ...
0
votes
1answer
28 views

Why is the p value by average marginal effects different than the p value of the coefficients?

In R, when fitting a logit regression model, why is the p value for variable X different when finding its average marginal effect (AME) (using logitmfx) than when finding variable X's p value by its ...
0
votes
0answers
20 views

Why is the alpha value set by the authors of the paper and not me, the reader?

Lets say I read a paper, I think that the data collection method and the measurement practice will introduce noisy data. The sample is small and the underlying hypothesis is somewhat grandios. ...
1
vote
1answer
29 views

How to calculate a p-vale from the t-distribution for a two-tailed t-test?

The resulting $t$ statistic from a t-test follows a t-distribution with parameter $k = $ freedom-degrees. For us to compute the probability of $t$ to occur, we use the cumulative distribution function ...
0
votes
0answers
25 views

Noninferiority test statistics for Logistic Regression

I have two groups of samples: standard care, and alternative. I want to compare a specific outcome (e.g., mortality) of the two, but I want to control for different covariates. So I run a logistic ...
1
vote
2answers
40 views

converting a bootstrap confidence interval into a p-value

If a bootstrap confidence interval (CI) can be interpreted as a standard CI (e.g., the range of null hypothesis values that cannot be rejected) [also stated in this post]. Is it ok to derive a p-value ...
1
vote
0answers
8 views

Why does p-value have to be defined as 'at least as extreme' as the test statistic rather than just the probability that T= t? [duplicate]

Say I am using a t statistic, the (two-tailed-t test) p-value as i understand is given as pr(t$\geq$|t| |null is true), why cant it be that pr(t=t|null) or at least some small relative frequency in an ...
1
vote
0answers
14 views

P-values and confidence intervals for Inverse Gaussian GLMM?

I want to test the effects of different variables on Reaction Times data. Following the recommendations of Lo & Andrews (2015) I compared the AIC/BIC of three GLMM with a Gaussian, a Gamma and an ...
0
votes
1answer
25 views

Which mathematical test do I need to use?

I have a dataset showing the yearly stock returns for two investment styles (growth and value investing) as well as a column showing the difference between the returns for each year. I have attached a ...
0
votes
0answers
17 views

Significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...
0
votes
0answers
15 views

Interpret repeated measures ANOVA results in R

I developed many model on groups of data I applied repeated measures anova to check the significant difference. Then I applied pairwise.t.test using Bonferroni correction. I got this result from the ...
0
votes
0answers
31 views

Help for interpretation : significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...
0
votes
0answers
30 views

How to get the p-value for a binary logistic regression model, not the individual predictors?

I ran a stepwise binary logistic regression model using glm in R. ...
1
vote
1answer
29 views

Interpreting the results of p-value and Cramer's V value

For my assignment I am working on a data set with the sample size n = 4.000.000 and about 450 columns. I have 4 rows in my cross table that have the values winter, spring, summer and fall. So my cross ...

1
2 3 4 5
41