Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

8
votes
3answers
6k views

Is p-value also the false discovery rate?

In http://surveyanalysis.org/wiki/Multiple_Comparisons_(Post_Hoc_Testing) it states For example, if we have a p-value of 0.05 and we conclude it is significant the probability of a false discovery ...
0
votes
0answers
12 views

Strange Tukey Output

I'm using HSD.test in package Agricolae to run a Tukey PostHoc test on heavy metals concentrations in otter livers and kidneys. I'm looking at differences between region, river basin, and age class. ...
0
votes
1answer
70 views

KS, Anderson-Darling and Cramer -Von Mises don'y work on these data . Why?

I am using Mathematica to model the distribution of the data below and I test the fits with AD, CVM and KS but none of them seems to work I was told that my data may have an excessive amount of ties. ...
0
votes
0answers
7 views

R GLMER Output - why isn't there information about the null model? [on hold]

With a glm() model there's information about the null deviance, which can be used to test whether the model significantly improves on the null. This isn't present ...
6
votes
3answers
473 views

Does independence imply conditional independence?

If two or more variables A, B, C, etc. are jointly mutually independent of one another, does this imply that that they are also conditionally independent given some set of conditioning variables X, Y, ...
4
votes
1answer
37 views

Can you statistically compare one subset of a population with the whole population?

I have data (total $n=90$) from a district of testing scores from $3$ subsequent years. I would like to compare MY school to the other schools, but I do not know which schools the other participants ...
1
vote
1answer
37 views

Formulating the likelihood in a particular likelihood ratio test

Here is a question from old exam papers I am having trouble with: Suppose the lifetime of a bulb is distributed exponentially with mean life $\theta$ (in hours). Let $X_i$ be the number of trials ...
0
votes
2answers
35 views

Which test you recommend?

Assume in a study the dependent variable is quantitative, while most independent variables are categorical, with some of them being quantitative. We aim to evaluate the relationship between the ...
5
votes
3answers
148 views
+200

A/B testing ratio of sums

Context Consider the following scenario for a company selling goods online. A user can purchase several items (i.e. basket of items), some of which are of particular importance and are tracked ...
0
votes
0answers
22 views

In normal hypothesis testing, why do we not use $H_0$ in construction of the estimate of $\sigma^2?$

In the normal hypothesis testing we teach in basic stats, why do we not use $H_0 = \mu_0$ in construction of the estimate for $\sigma^2$? E.g., $\frac{1}{n}\sum_{i = 1}^n(x_i - \mu_{0})^2$? It would ...
0
votes
1answer
44 views

Asymmetric data analyses

I have a data set containing bacterial count values before treatment and after treatment. These data look asymmetric. Please advise me for a suitable statistical test to find if the introduction of ...
1
vote
1answer
181 views

Compare cumulative distribution functions of several function point counters

Function point counters are used to measure the size of software. However, they are believed not to deliver exact results. Different measurements sometimes produce different results. The performance ...
2
votes
1answer
29 views

AB Testing vs Hypothesis Testing

I'm having trouble making the full connection between AB Testing and Hypothesis testing. Hypothesis Testing Conducting a test to compare two means, we need the standard deviation of both samples, and ...
1
vote
0answers
18 views

Testing for non-zero values above and below the diagonal of a matrix

I have a $n\times n$ contingency matrix comparing count data (classification) of two procedures. I want to test for non-zero values above and below the diagonal (different classifications in both ...
0
votes
0answers
9 views

Mixed logit with repeated choices coefficient comparison

We are analysing data from the discrete choice experiment using mixed logit model. Data is panel, since we had repeated choices. We would like to compare if the difference in coefficients within the ...
0
votes
0answers
4 views

What is the difference between Wald and Wilk tests?

I know that the Wilk test uses the estimate obtained from MLE, but intuitively I do not understand the difference between the 2 tests. When shall I use one or the other? How do the results differ?
0
votes
0answers
11 views

Best Test for Limited Data

I'm just getting my start in learning statistics, and I'd love to be able to apply some of the tools I'm learning to my professional work. Sometimes I find it hard to determine how to apply that ...
1
vote
1answer
180 views

How to study the cause and effect relationship between 2 variables?

I have data from a random sample of hotel bookings. I want to study the cause and effect relationship between number of days in advance that the hotel was booked (or book lag, also called ...
1
vote
0answers
190 views

Johansen Test - What is a lag?

I am new to the Johansen Test, have been using the ADF test but due to its limitations I am trying to switch over. I have taken data (1 hour bars) for 4 symbols and passed it through this function ...
0
votes
0answers
7 views

Test for unequal correlation coefficients inflates Type I error for non-uniform sample distribution. How to correct?

Aim I'm studying the hypothesis test which estimates the probablility of the null hypothesis: The difference between two correlation coefficients is 0. More exactly, I deal with the question ...
0
votes
0answers
6 views

Exponential Test

Given a collection of data points, without any other information I want to test for an exponential distribution. In a different case, I tested for normality using an Accord implementation of the ...
0
votes
0answers
17 views

T-Test Raw Data vs Sampling Data with CLT

I have some data that I want to run a T-Test on and get a p-value of 0.05. See below: I then using the CLT take 30 samples from each sample and take the mean of each sample. I do this 100 times to ...
0
votes
0answers
29 views

R: Interpreting the emmeans' t-tests [duplicate]

Since the df are different between t-tests and emmeans ones (cf. https://stats.stackexchange.com/a/369671/217618), how then should I interpreted the results?
0
votes
1answer
100 views

Create an A/B Sample Size Calculator using Evan Miller's Post

To learn more about A/B Testing sample sizes selection I am attempting to use Evan Miller's popular blog-post to recreate a sample size calculator (https://www.evanmiller.org/sequential-ab-testing....
0
votes
0answers
35 views

How do I set up the data for a Friedman Test?

Below is a small sample of a very large data set (over 10,000 records). All I'm trying to do is determine if there's a statistically significant difference found in the ordering of supplies among ...
3
votes
0answers
486 views

Is there an error in this published Chi-squared table?

I am questionning the Chi-squared table given in 100 Statistical Tests (3rd Edition, 2006) by Gopal K. Kanji, reproduced in part below. Specifically, I think the Two-sided and One-sided levels of ...
0
votes
0answers
10 views

Multiple comparisons, should I run a one-way multivariate analysis or can I use Mann Whitney?

I am analysing some study data and am unsure the appropriate statistical test to use. The study involves watching a film. 50% of participants saw version 1, 50% saw version 2 - (the difference in ...
1
vote
1answer
182 views

Version of Mann-Whitney U test, for right-censored data

Is there an equivalent of the Mann-Whitney U test, but for right-censored data? I have right-censored numerical data from two different groups. The mean for one group is higher than the other. I'd ...
0
votes
0answers
18 views

Kolmogorov Smirnov test and dependence on number of parameters

Somewhere I heard that Kolmogorov Smirnov test depends on number of parameters, is true? If yes, so I have problem because I have used ks.test(...) in R for normal distribution with 2 parameters and ...
0
votes
1answer
23 views

Finding the uniformly most powerful test for hypothesis

Let $\mathbf{X}=(X_1,...,X_n)^T$ is a simple sample where $X$ belongs to exponential distribution family $\mathcal{P}=\{ f(x;\mu,\sigma \}, -\infty<\mu<\infty, 0<\sigma<\infty.$ Density is ...
-2
votes
0answers
24 views

In hypothesis testing are we not being explicitly partial? [on hold]

Semper necessitas probandi incubit ei qui agit! https://en.wikipedia.org/wiki/Occam%27s_razor Stipulates that science should be based on ignorance the lack of belief. https://en.wikipedia.org/wiki/...
2
votes
1answer
85 views

Bootstrap for goodness of fit

Suppose I want to test whether a die is fair. I toss 60 times with outcome $X = (40,20,0,0,0,0)$ where $i$th coordinate denotes times of $i$th face occurs.Consider Hypothest test($\alpha = 0.05$): $$...
1
vote
1answer
182 views

Compare proportions from dependent samples

I have two groups of observations, each observation can belong to positive or negative case. For each group I calculate percentage of positive cases. I want to check whether difference of positives ...
26
votes
4answers
20k views

Measures of similarity or distance between two covariance matrices

Are there any measures of similarity or distance between two symmetric covariance matrices (both having the same dimensions)? I am thinking here of analogues to KL divergence of two probability ...
0
votes
1answer
37 views

How to test if 2 proportions from the same sample are significantly different

I asked a random group of people from an organisation a question. I then have a proportion of favorable responses (e.g. 80% of 100 participants gave favorable responses). 6 months later, I asked ...
0
votes
0answers
15 views

Shapiro–Wilk test for non-normal distributions

The Shapiro–Wilk test makes use of the correlation between the data (a.k.a. sample quantile) $x$ and its theoretical quantile $m$. As far as I'm concerned, it's a normality test simply because the ...
0
votes
2answers
185 views

Independent or Dependent Samples

I want to perform a statistical test: Wilcoxon Signed Rank Test to test if there is a significant difference between them, but I don't know if the two samples are dependent or not? Context: I have a ...
1
vote
1answer
189 views

Interpretation of the t.test and Wilcox.test in R

I got two distribution which are not pairs. It's difficult to say that they are also from a normal distribution because they have different length. The distributions of the vectors are, for gas: <...
2
votes
1answer
187 views

Null hypothesis of t-test and ANOVA

I've recently started to brush up my statistics, and have begun doing so with two of the most often used statistical tools in my field: the t-test and ANOVA. Reading up on popular information sources ...
0
votes
0answers
18 views

A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
0
votes
1answer
34 views

Question about statistical test for classification problem in a very unique case!

I'm working on a non-standard dataset with deep learning, and I want to prove that my method is better. We have a dataset composed from dataset A and dataset B where each one is composed of ...
0
votes
1answer
183 views

AB test, but, more like ABCDEFGH

When doing an AB test (for website variations), I understand you should check the significance before drawing conclusions with any of the tools you can find googling "ab test calculator", now how ...
0
votes
0answers
9 views

How to write up a Tukey Post Hoc Finding- please

How do I interpret these results into a Post Hoc Tukey HSD? TIA.
0
votes
1answer
17 views

Computing the power of Welch's t-test

I constructed a Welch's t-test in order to examine the difference between the means of two samples (check whether or not one is larger than the other). $H_{0}: \mu_0 = \mu_1 ~\textrm{vs}~ H_{1}: \...
1
vote
0answers
14 views

Comparing AUC between two subsets of test-fold

I suspect that the out-of-sample AUC of my model depends on the number of events in the out-of-sample/testing fold (I am modelling a binary variable, 0/1). In order to test this hypothesis, I want to ...
0
votes
0answers
34 views

Should we use the hypothesized population variance when calculating the CI even after rejecting the null hypothesis?

Suppose we want to verify the probability a sample comes from a distribution with mean $\mu_1$ and variance $\sigma²_1$. Then, we conduct a hypothesis test and decide to reject the null hypothesis, i....
35
votes
4answers
12k views

Bayesian equivalent of two sample t-test?

I'm not looking for a plug and play method like BEST in R but rather a mathematical explanation of what are some Bayesian methods I can use to test the difference between the mean of two samples.
13
votes
2answers
6k views

Using statistical significance test to validate cluster analysis results

I am surveying the use of statistical significance testing (SST) to validate the results of cluster analysis. I have found several papers around this topic, such as "Statistical Significance of ...
-1
votes
0answers
12 views

Comparing two classifier with non-normal accuracy distribution

I'm working on a non-standard dataset with deep learning, and I want to prove that my method is better by comparing accuracies of my classifier and that of a baseline. I used ANOVA and Wilcoxon but I'...