In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

learn more… | top users | synonyms (2)

1
vote
0answers
26 views

Do I need to divide p value by two?

I have used LMM with this formula: f1 <- lmer (dprime_f ~ language_f + (1+language_f|listener_f), data = data1.frame, REML = TRUE) Then I used lsmeans to run ...
2
votes
2answers
31 views

How to interpret a VIF of 4?

I am doing a multiple regression, trying to test the extent to which personal income changes and Presidential popularity can predict election results. I have a small sample size, unfortunately, as ...
0
votes
1answer
21 views

Significance (p-value) in regression/size of parameter estimate

In a linear (OLS) regression, I find that the sizes of the coefficients vary a lot: one of them is about 0.0005 and another is ~150. The p-values are all under 0.1. Is this reliable result? These are ...
0
votes
0answers
8 views

Comparison between coefficients of 2 polynomial fits

I have a cancer data according to men and women. I fit cancer incidence polynomially to men and women separately based on year. And I get 2nd order year coefficients for men as -3.04e-4 and -6.10e-4 ...
2
votes
0answers
19 views

G-test vs Pearson's chi-squared test

I'm testing independence in an $N \times M$ contingency table, I don't know which of the G-test or Pearson's chi-squared test is the best ? The sample size is in the hundreds but there are some low ...
10
votes
2answers
334 views

Is rejecting the hypothesis using p-value equivalent to hypothesis not belonging to the confidence interval?

While formally deriving the confidence interval of an estimate, I ended up with a formula that resembles very closely the way $p$-value is computed. Thus the question: are they formally equivalent? ...
0
votes
0answers
12 views

Significance test for difference between subset mean and population mean

I have customer usage data - number of products purchased by each customer during a specified time period & customer attributes (gender, location etc.) I want to make statements like male ...
0
votes
0answers
9 views

Significance test for subset variation from population

I have customer churn data: independent features like customer gender, location, age etc. outcome feature representing whether he/she has churned or not My population churn rate is around 30% ...
0
votes
1answer
14 views

Can adding uniform sampling weights change p-values?

I am doing an analysis of Average Marginal Component Effects (AMCEs) of a conjoint experiment in R. The package I use, cjoint 2.0 (see amce function), allows to add sampling weights (I want to compare ...
0
votes
0answers
18 views

Bootstrap pvalues: strange adjustment — pbkrtest PBmodcomp LRT

As far as I can tell, PBmodcomp is doing some straight up bootstrapping, similar to this Faraway example for lmer models mmod and rmod (maximal and reduced models): ...
0
votes
2answers
61 views
0
votes
1answer
39 views

P-Values decrease when additional significant variables added (multicollinearity?)

I am doing a study for my masters correlating two separate development indicators to election results for the incumbent government. Unfortunately, I was only only able to get 11 years worth of data ...
0
votes
0answers
40 views

Test of Equal Proportions with exact p-values [duplicate]

The question "Test of Equal Proportions with zero successes" showed me that using a test of equal proportions with very low or very high numbers of successes doesn't give an accurate p-value. Below is ...
0
votes
0answers
41 views

Z-test and confidence intervals for mutual information

Given two discrete random variables $X,Y$, I would like to obtain confidence intervals for the Mutual Information $I(X;Y)$ and a p-value for the null-hypothesis that $X,Y$ are independent. I would ...
3
votes
1answer
40 views

Combining very small p-values - Fisher's method gives 0

I have 30 p-values all very small (varying between $10^{-140}$ and $10^{-110}$), and I want to combine them in some way to get a single statistic. I learned about Fisher's method, but if I apply it to ...
1
vote
2answers
29 views

Comparing 2 methods based on 2 sets of p-values

I have a method A that gives me 3 p-values (say p1A, p2A, p3A) for measuring 3 performance metrics. I have a method B that gives another set of 3 p-values (say p1B, p2B, p3B) for measuring the same ...
2
votes
1answer
50 views

Regression Analysis: R squared and p-value

I would like to know if the coefficient of an independent variable is still relevant if the R-squared is low (assuming the p-value for the independent variable is less than 0.05). For example, ...
4
votes
1answer
55 views

What is the reason for the shape of the volcano plot?

I use it in biostatistics. For example, why do genes (or any input value) with a larger absolute fold-change also tend to have a more significant p-Value? All volcano plots look like it, so I assume, ...
4
votes
0answers
64 views

What are the reasonable properties of p-value combination methods when all p-values are the same?

Suppose we test a null hypothesis $H_0$ by combining evidence from $n$ independent tests through some $p$-value combination method $M$. The combined $p$-value is $M(p_1,...,p_n)$. Assume the extreme ...
1
vote
1answer
25 views

P-values for regression coefficients in total least squares regression

I want to calculate the p-value for the beta estimated in Total (orthogonal) least squares regression. Do I need to calculate the standard error of the estimates in ...
3
votes
0answers
44 views

Is the likelihood a valid statistic to assess p-value?

The p-value is defined as the probability, under the assumption of hypothesis H, of obtaining a result equal to or more extreme than what was actually observed (Wikipedia). By "a result" it is ...
4
votes
3answers
97 views

Does false discovery rate depend on the p-value or only on the alpha level?

Let's say I get a p-value of 0.001. I know that alpha level dictates the probability of a type I error, so if I get a result this significant, is my false discovery rate (FDR) lower than if I were to ...
1
vote
0answers
23 views

Meta-analysis using p-values

I have a study-level (quasi-experimental) estimator of treatment effect $\hat{\delta}_i$, where $i$ indexes the study. Due to the way the estimator is constructed, I do not have access to a measure of ...
1
vote
1answer
90 views

Test if two normally distributed random variables have the same mean

We have two independent random variables which follow normal distributions $X_1\sim \mathcal N(\mu_1,\sigma_1)$, $X_2\sim \mathcal N(\mu_2,\sigma_2)$. From the context, we have that $\mu_1\leq\mu_2$. ...
4
votes
2answers
306 views

Misunderstanding a P-value?

So I've been reading a lot about how to correctly interpret a P-value, and from what I've read, the p-value says NOTHING about the probability that the null hypothesis is true or false. However, when ...
4
votes
0answers
61 views

Cherry picking and low p-values

Let's say I run a lot of univariate OLS regression models, say 200,000, with 50 data points, then cherry pick the best one (highest r-square). If my $p$-value for this model is way less than ...
3
votes
1answer
141 views

Trying to draw conclusions from p-values

I am still trying to figure out how one can appropriately interpret p-values, and I am running into a bit of an issue with some of the data that I just finished analyzing. I have two types of plots ...
1
vote
0answers
14 views

Multiple test correction procedures for multiple regression with dependent predictors

A researcher is employing OLS multiple regression to examine the independent effects (i.e., partial correlations) of a moderate (~8) number of theoretically-relevant predictors on some outcome. The ...
2
votes
1answer
45 views

What happens when there's a one-tailed hypothesis test and the results are in the other tail?

For example, let's say I'm using a one-tailed t-test to see if a box of Lucky Charms cereal has more than 100 marshmallows. I know the consequences of using a one-tailed t-test and how it wouldn't be ...
0
votes
0answers
22 views

Multiple tests, summary of clogit

I have 10 exposure variables that i would like to analyze univariate. I've applied the conditional logistic regression model for each exposure and then used the summary function for each one of them. ...
0
votes
0answers
22 views

How do one interpret the results of wilcoxon-mann-whitney test in SAS?

I have done the Wilcoxon test in SAS. My class variable is race. I got a low p-value. What exactly does that mean?
0
votes
1answer
26 views

Data analysis: how to compare 2 variables from each subject across 3 groups of 200 subjects?

New here and not a statistician, am MD doing research. Need advice on best method for data analysis. I have 3 groups of 200 subjects each. We collected 2 data points (A,B) from each subject. Baseline ...
0
votes
1answer
35 views

(Dis)Advantages of correlation vs. $R^2$ vs. p-value of linear regression for two variables?

I would like to know what are advantages and disadvantages of $R^2$ vs. correlation (e.g. cor() in R) vs. p-value of linear regression for two variables/features? ...
0
votes
1answer
13 views

Trying to migrate from P values to CI… how to know when to reject null hypothesis? [duplicate]

So, I know that if we get a p<= 0.05 (95% CI), I know that the null hypothesis can be rejected. Now my question is how do I do that with CI? Eg. in the table below A17S group has a CI of ...
4
votes
1answer
55 views

Why does Stouffer's method work?

It seems like a fairly straightforward question, but when I really think about it, Stouffer's method doesn't make sense to me. This is why: Assume a two-tailed hypothesis. You first calculate $z_i$ ...
1
vote
0answers
33 views

Adjust p-values for multiple comparisons

I want to evaluate if a proposed modification M* to a base classifier M is better in terms of accuracy. Both, the base classifiers M and their respective modifications M* are tested on N datasets. To ...
0
votes
0answers
20 views

Obtaining significance for mixed GLMMs on count and binary data

I'm new to the software R and am trying to compute statistics on data from experiments on the offspring of lizards from two different thermal treatments - looking specifically at differences in their ...
1
vote
1answer
24 views

Coefficients of Categorical P-Values in Regression - Condense or Apply 0?

This analysis is part of a much larger research project, but I've extracted a simple example that will fit the question I have. Y = B0 + B1 * [Day of Week] + E Day of Week is coded as 1 (Sunday)-7 ...
0
votes
0answers
5 views

Compute p-value from foldchanges taking into account level of expression

I've expression values for 1 gene for ~50 samples and I've a sample within my 50 samples, that harbours a specific alteration into this gene (i.e. the mutSample). The other 49 samples do not harbor ...
0
votes
1answer
21 views

Propensity score stratification: standard errors and p-values

While there are many tutorials on how to perform propensity score stratification, I was unable to find any example that showed the calculation of standard errors and p-values for the final estimate. ...
0
votes
0answers
39 views

What does the P value in restricted cubic spline plot mean? [duplicate]

What does the P value in the following example mean? ...
8
votes
1answer
135 views

Fisher's exact test gives non-uniform p-values

I am trying to apply Fisher's exact test in a simulated genetics problem, but the p-values appear to be skewed to the right. Being a biologist, I guess I'm just missing something obvious to every ...
1
vote
0answers
16 views

Meaning of $\psi$ in regard to $p$-value distributions

In Combining dependent P-values by Kost & McDermott, the notation $\psi$ is used in a way that I am not familiar with. Relevant passge: 2.1. Known Variance. Let ...
2
votes
2answers
47 views

How can you have p < 0.0001 with a sample size of 89 and a control group of 52?

From Efficacy of monoterpene perillyl alcohol upon survival rate of patients with recurrent glioblastoma (linked from Does Frankincense cure cancer?): Patients and methods It was included 89 ...
78
votes
9answers
12k views

Is this really how p-values work? Can a million research papers per year be based on pure randomness?

I'm very new to statistics, and I'm just learning to understand the basics, including $p$-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. ...
0
votes
1answer
26 views

How can I extract all the p values from a model for an individual factor variable?

Let's say I have factor variable Var_1 with 3 levels, "1", "2", and "3". "1" is the base level of the factor variable. So when I run a linear regression, I get a p value for each level of that ...
1
vote
0answers
20 views

Calculate p-values based on many distributions without knowing which the test statistic is from

I have many (20) normal distributions of the probability of a certain event occurring. Each of these 20 has n>1000.I have an observed proportion from a separate sample that I want to compare to ...
0
votes
1answer
23 views

Correlogram q-statistics of residuals

I am currently try to get information from the correlogram of residuals in eviews from a certain equation; I am supposed to understand if residuals are white noise or not and to adfirm that they are ...
3
votes
0answers
48 views

Two sample one-sided Kuiper Test and KS-statistic

With the KS-Test it is possible to conduct a two sample one-sided test between two different random samples $A$ and $B$ to test whether one CDF is larger or smaller than the other, i.e. is $CDF_A$ ...
2
votes
2answers
95 views

Why does the p-value double when using two-tailed test compared to one-tailed one? [duplicate]

Since this has been marked as a duplicate, I want to clarify that it's not about critical values of one- vs two-tailed tests, but the calculation of the p-Value in this case. I get that in a ...