Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

563 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
169 views

Efficient nonparametric estimation of confidence intervals and p-values for nonlinear regression

I'm estimating parameters for a complex, "implicit" nonlinear model $f(\mathbf{x}, \boldsymbol{\theta})$. It's "implicit" in the sense that I don't have an explicit formula for $f$: its value is the ...
7
votes
1answer
78 views

Why does Bayesian p-value involve the parameters in addition to the data?

On page 146 of Gelman's Bayesian Data Analysis, Gelman discusses Bayesian p-value as a way to check the fit of the model. The idea is to compare the observed data ($y$) with data that could have been ...
6
votes
1answer
957 views

Get p-value of coefficients in regression models using bootstrap

I've been reading a lot these last few days about how to get a p-value from bootstrap for regression models (not by permutation). For each coefficient of the model, the null hypothesis is that the ...
6
votes
0answers
1k views
6
votes
0answers
575 views

Skew of p-value distribution under composite null hypotheses

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
5
votes
0answers
59 views

Non-uniform p-values from hoeffd function in Hmisc when data sets are independent

When using the function hoeffd in the CRAN package Hmisc I get unusual p-values for pairs of data sets that are independent. The function hoeffd is an implementation of Hoeffding's $D$ statistic. ...
5
votes
0answers
1k views

Use Chi-Squared or Binomial Test if Distribution is not Known?

Suppose you have a set of data (eg. [a, b, a, a, b, b, etc.]), and you have the suspicion that the set of data follows the binomial distribution. Your Null Hypothesis is: The probability of success ...
5
votes
0answers
599 views

Confidence-interval / p-value duality vs. frequentist interpretation of CIs

Many sources suggest that there is a duality between confidence intervals and hypothesis testing.(*) But I'm having trouble making sense of this philosophically. The frequentist interpretation of a ...
5
votes
0answers
81 views

Analysis of “The advantage of short paper titles”

After seeing the paper, The advantage of short paper titles showing a relationship between paper length rank and citation rank, I wondering what to make of the results. As a layman, my questions are: ...
5
votes
0answers
758 views

Using the hypergeometric distribution for skipping events in transcriptome sequencing

My question is inspired by this post. However, its a bit more complicated than that to explain. I hope I succeed. I work with RNA-Seq data on alternative splicing in plants. For this discussion, lets ...
4
votes
1answer
80 views

Why is the population standard deviation approximated as the sample standard deviation?

This question addresses calculating a p value from the mean and standard deviation statistics of a sample. I understand that the -general- philosophy is to divide the sample standard deviation by the ...
4
votes
0answers
135 views

Comparing multiple frequency distributions from data and from simulation

I am attempting to analyze the shuffling in the card based online game Magic Arena, which I suspect has a specific bug that causes the distribution of outputs to be biased. Is the approach I have ...
4
votes
0answers
82 views

P-value: Fisherian vs. contemporary frequentist definitions

I am trying to see if I understand the definition of $p$-value as used by Sir R. A. Fisher and the one used today by frequentist statisticians (not sure how to call it better). $p$-value according ...
4
votes
0answers
195 views

Do results from the lme-function require adjustment of p-values?

If I run a linear mixed model with the lme() function and get results like these (comparing the score of 4 treatment groups against a placebo group): ...
4
votes
0answers
32 views

How does Lindley compare a Bayes factor and a p-value?

I was reading this paper by Dennis Lindley ("Analysis of a Wine Tasting", J. Wine Econ. 2006). Statistically, the paper is a straightforward analysis of a $10\times 11$ two-way table. To test whether ...
4
votes
1answer
84 views

Statistical test to identify enriched edges in a network

I have a network with N nodes and E directed edges. Each edge (Eij) is an integer that represents the number of connections between a source node (Ni) and a target node (Nj). My null model is that ...
4
votes
0answers
87 views

Determine p-value if $\Theta_0$ is not singleton

Here's what I've learned about general test problems: Let $X_1,\ldots X_n$ be random variables whose distributions depend on a parameter $\theta$ in some parameter space $\Theta.$ A general ...
4
votes
0answers
226 views

Reliability of p-value in scipy.stats.spearman

I am puzzled about the following sentence in the documentation of the function calculating Spearman's rank correlation coefficient in the scipy.stats package: "The p-values are not entirely ...
4
votes
0answers
285 views

Is this popular approach to Bootstrap hypothesis testing correct?

A widely cited paper "Statistical Significance Tests for Machine Translation Evaluation" proposes usage of Bootstrap for evaluating significance of difference across systems for Machine Translation (...
4
votes
0answers
122 views

Determining statistical significance in difference between two CRF models

I have trained two Conditional Random Field (CRF) models Ma and Mb in CRF++, the first using a simple set of features, the ...
4
votes
0answers
1k views

lmPerm p-values and multiple testing

I've started using lmPerm in order to perform regressions in R. The equation I want to fit has the form: ...
3
votes
0answers
116 views

gam() p-value seems to be unreliable - why and how?

I'm investigating the effect of overhead cover (tree canopy) on the proportion of birds scavenging on carcasses left out in nature. These data are taken from different national parks with different ...
3
votes
1answer
38 views

Benjamini-Hochberg procedure with equally ranked p-values

I struggling with Benjamini-Hochberg correction due to equally ranked p-values and I would gladly appreciate some help. My problem is as follow: I've got 11 comparisons to do and the threshold is set ...
3
votes
1answer
44 views

Can I say one models Spearman coefficient is better than that of another model based on their respective p-values?

Is it possible to determine whether one model's Spearman rank is significantly better than that of another model based on the p-values for each? Let's say that I have model 1: $\rho_s$ = 0.9, p-...
3
votes
0answers
62 views

Are these equivalent (for p-values): super-uniform, stochastically larger than / dominating the uniform, conservative?

In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is ...
3
votes
0answers
75 views

Does a high Chi square p-value for a whole model mean it is insignificant if likelihood ratio tests indicated variables should be added?

I've been estimating lots of versions of the same model by incrementally adding variables. With some variables, if I add them to the model, the likelihood ratio test indicates that they are ...
3
votes
0answers
58 views

An alternative definition of the $p$-value

As we know, $p$-values are uniformly distributed under $H_0$. This urges me to ask if this constitutes a valid (re-)definition of the $p$-value. $p$-value - A statistic with a uniform distribution ...
3
votes
0answers
48 views

Has the reproducibility crisis affected confidence intervals as well?

The reproducibility crisis has given many pause over the value (?) of $p$-values to measure the relevance of statistical findings. Given the interpretation of a $p$-value and some knowledge of ...
3
votes
1answer
79 views

Bootstrap Confidence Interval, f-value and p-Value

I am trying to understand the intricacies of Linear Regression (1 predictor, 1 response). I am using Istanbul Stock Exchange dataset (https://www.kaggle.com/uciml/istanbul-stock-exchange) to do the ...
3
votes
0answers
209 views

Bootstrapped post-hoc analyses for mixed effects models

I am using Linear Mixed Effects modelling (LMM) to test the effect of a single predictor 'Age' (a 4 level factor: "-1", "1", "3", or "8") on a logit-transformed response variable (percentage herbivory)...
3
votes
0answers
702 views

Overall p-value for zero-inflated beta regression mixed model

I am analysing vegetation percentage cover data from grazed and ungrazed plots in R using a zero-inflated beta regression in package gamlss. Here are some example ...
3
votes
0answers
393 views

p-values for permutation tests

Let's say I want to test whether the average performance of two individuals who work together (let's say in the same room) is different to their performance when they would not have worked together. ...
3
votes
0answers
7k views

How to find p value using estimate and standard error?

I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models For example, consider two independent studies with effect estimates and standard errors ...
3
votes
0answers
616 views

How to calculate a p-value under the null from a bootstrap?

In another thread, I asked how to obtain a confidence interval of the difference in probability of success between two groups from a logit model and @Weihuang Wong showed me how to do this with a ...
3
votes
0answers
73 views

When does it make sense to treat a time series as a non-time series?

Does it ever make sense to treat a time series as not being a time series? For example, I have a time series of nine years of annual summer averages of zooplankton biomass and cumulative lake inflow. ...
3
votes
0answers
81 views

Unstable p-values in logistic regression

I'm tweaking a logistic regression model in R. The model's been optimized and tested pretty extensively so far. I need to remove one variable for non-statistical reasons (...
3
votes
0answers
155 views

p-values and predicted standard errors using cplm

I am using the cpglmm-function from the "cplm" package to model the effect of treatment, time and host age on logged parasite counts. This type of data has a big peak at zero, but follows a ...
3
votes
0answers
99 views

R : qvalue package - smoother or bootstrap method for pi0 estimation

I've a bunch of 500 p-values and I want to estimate the pi0 (proportion of true H0). When using the qvalue R package, two method can be used (smoother or bootstrap). Both giving very different results ...
3
votes
0answers
95 views

Correct Sample Space for Calculating P-Value

Suppose there are two variables, x and y, and we wish to calculate some test statistic (say the Pearson correlation), and estimate its significance. The pvalue is calculated by estimating the ...
3
votes
0answers
505 views

p-value distribution and the applicability of different multiple testing correction methods

I came across this issue as I was analysing some genomic data sets looking for differential DNA methylation. I have generated p-values from the comparisons I am making, and am now looking into the ...
3
votes
1answer
147 views

confidence interval in quadratic programming

This is a quadratic programming problem coming from constrained linear regression $A b + \varepsilon = y$ and $B^T b >0 $ then minimize $\varepsilon^T \varepsilon=(A b - y)^T (A b - y)$ which is ...
3
votes
0answers
643 views

Including covariates makes QQ plot worse

I know this is similar to this question, but that one didn't seem to get a satisfactory answer. I'm using plink to run a GWAS. My phenotype data are binary, so it's performing a logistic regression ...
3
votes
0answers
198 views

Is the likelihood a valid statistic to assess p-value?

The p-value is defined as the probability, under the assumption of hypothesis H, of obtaining a result equal to or more extreme than what was actually observed (Wikipedia). By "a result" it is ...
3
votes
0answers
68 views

Meta-analysis using p-values

I have a study-level (quasi-experimental) estimator of treatment effect $\hat{\delta}_i$, where $i$ indexes the study. Due to the way the estimator is constructed, I do not have access to a measure of ...
3
votes
1answer
49 views

Correlation Analysis of Standarized Testing Data

I need some insight on how to correctly analyze some data using correlations. The nature of the data and research endeavor is as follows: I have two independent samples of different sizes that ...
3
votes
0answers
1k views

How to test the significance of covariance?

I'm using the Mutual Informacion covariance in RNA sequences and I want to know if there exits a way to test if some covariance is significant, let's say, an associated p-value. Thanks to all for ...
3
votes
0answers
408 views

Basic methods for detecting outliers

Let $X$ be a matrix of $n$ rows and $m$ columns. $n$ is the number of samples and $m$ is the number of gene expressions. Gene expressions are basically numerical continuous values. Assume we have a ...
3
votes
0answers
3k views

Converting Chi Square Adjusted Residuals to p values

I am running a chi squared analysis in SPSS on a 5 x 2 contingency table. The overall chi square p value was < 0.0005, so I ran post hoc tests (adjusted standardized residuals on the individual ...
3
votes
0answers
730 views

How to calculate p-value using an algorithm on R assuming that distribution is unknown

For a given sample x, where x is distributed with a normal distribution mean known but variance unknown. I am testing the hypothesis that variance is equal to one or greater than 1. The question is to ...
3
votes
0answers
75 views

Running sequential tests as data is collected - practical advice for researchers

I've observed that many experimental labs in my field don't do a power analysis before they start collecting data. In practice, it seems that many researchers collect data and periodically run ...