Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

4
votes
1answer
60 views
+50

On a Bayesian hypothesis testing

Let $X_1,...,X_n$ be a random sample and $\lambda >0$ be a parameter, with $X_i |\lambda \sim Poisson (\lambda)$ and $\lambda \sim Gamma(\alpha, \beta) (\lambda)=\dfrac {1}{\Gamma(\alpha) \beta^\...
2
votes
1answer
27 views

Transitivity of statistical significance

Say I have three sets of observations, $A$, $B$, and $C$ (say, the heights of Australians, Americans and Brits). I assume that they are all normally distributed, independent etc. I perform two-sided ...
1
vote
0answers
18 views

Understanding equivalence testing. Why there are two hypothesis?

I have to two groups and I would like to test that difference between means of these two groups is at least 0.02. I am using TOSTER R package. Here are details: ...
0
votes
0answers
9 views

How to calculate studentized bootstrap pvalue for test with H0 assuming lift (%)?

First of all I'd like to mention, that I've read at least 2 great discussions on the site, which helped me a lot. Very thorough explanations of H0 data generation are presented here and here. But ...
0
votes
0answers
6 views

Confidence interval of one tailed Z-test

In an intelligence test of boys and girls, the question has asked to find if the boys and girls are equally intelligent against the alternative that girls are more intelligent than boys. While I have ...
0
votes
0answers
6 views

Sound unit testing of convergence of an implementation of a stochastic algorithm

(My statistics knowledge comes mainly from computer science, so I might be overthinking the following...) I have implementations of several stochastic optimization algorithms (evolutionary algorithms,...
0
votes
0answers
9 views

Break down a 2-by-3 within within ANOVA into two 2-by-2 within within ANOVA

I have data that may be arrayed as a 2 (Time: A1 vs. A2) by 3 (Difficulty: B1, B2, B3) within-within factorial. But I'm not interested in the comparison of difficulty B2 and B3 over Time. Is it ...
1
vote
1answer
29 views

Logical Hypothesis Modification

I performed an experiment with a treatment and a control group. It just so happened that my treatment could be modified to either increase or decrease my dependent variable. So, I decided to test two ...
2
votes
1answer
64 views

Wilcoxon signed-rank test or One Sample t-test?

I conducted a training session and for each of my users, I measured pre and post score (before and after training session). Each of my users (n=28) is measured twice, so my data is paired: Data: <...
0
votes
0answers
13 views

What statistical to use - variability between but not within

I have a data set where each line represents a participant's response to a series of questions (e.g., attitudes towards democracy, life satisfaction). Participants were recruited from different ...
1
vote
0answers
33 views

Replicating Dickey Fuller? How did they know something was wrong? [cross posted on QF] [on hold]

I am interested in testing if there is size distortion through simulations. I have recently been interested in replicating Dickey and Fuller (1979) and this source from another post helped a lot, here ...
1
vote
1answer
69 views

Setting up contrasts in lmer?

0 I have 139 subjects (ID), with measurements taken at two time points (Time1, Time2), at 148 brain regions, a dependent measure called volume, and a covariate called thickness. Each subject has 148 ...
8
votes
5answers
1k views

Why do we need alternative hypothesis?

When we do testing we end up with two outcomes. 1) We reject null hypothesis 2) We fail to reject null hypothesis. We do not talk about accepting alternative hypotheses. If we do not talk about ...
2
votes
1answer
34 views

Why replication studies use two-tailed tests?

I have been searching replication studies and I found that all of them use two-tailed tests. For example: Open Science Collaboration. (2015). Estimating the reproducibility of psychological science....
0
votes
0answers
14 views

Information gain or regression for measuring to what extent a continuous quantity detemines a discrete one?

I have a small population S and two quantities P and Q associated to its members. P is integer valued (say, age taking integer values between 1 and 20) and Q real valued (say, height in meters). I can ...
0
votes
0answers
26 views

One-sided variance test in r [duplicate]

I would like to compare variances of two independent samples (of different sizes) from two unknown distributions. It seems I should use a one-sided nonparametric test. Bartlett's and Fligner's tests ...
0
votes
0answers
21 views

chi square testing visualization [on hold]

How can I plot something like the first plot, I do have only one sample called s_chi in order to get statistical chi and plot the hypothesis testing my code:
0
votes
1answer
125 views

Friedman's test or Kruskal-Wallis?

I am investigating the effect of an intervention treatment on a group of samples over 5 time points (a period of 2 months in total) For example: Subject 1 - Timepoint 1 (Dose 1), 2 (Dose 1), 3 (Dose ...
0
votes
0answers
22 views

test of relation when one variable is itself a mean?

What is an appropriate test to see if there is a relationship between two things, where one of them is itself obtained by averaging human rankings. As an example, there are good and bad bottles of ...
1
vote
0answers
27 views

Making Inferences from a sample

I am trying to teach myself basic inferential statistics by going through online video lectures. However there are some gaps in my core concepts. I have a couple of quick questions Assume I take a ...
-1
votes
0answers
13 views

How to test the homoskedacity using GQ test, Interpret R2 with VIF, Calculate Adjusted R-squared [on hold]

Can you please show me with some explanations how to test the homoskedacity using GQ test, calculate and interpret R2 with VIF for a model, Calculate Adjusted R-squared from the pictures from this ...
1
vote
1answer
329 views

R-squared or ICC or Kappa or pairwise t-test comparisons to compare ratings

I would like to compare the scores (0-100 scale) of students who are rated by three different examiners. The correct score is assigned by fourth examiner. I would like to quantify how different the ...
1
vote
0answers
24 views

What test should one use when dealing with different treatments in plant clones?

In an experiment involving plant clones there are: 20+ treatments, 5 technical replicates for each treatment (plant clones in which the treatments were tested). Variable of interest is Biomass. Due ...
19
votes
2answers
938 views

Why doesn't Wilks' 1938 proof work for misspecified models?

In the famous 1938 paper ("The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Mathematical Statistics, 9:60-62), Samuel Wilks derived the asymptotic ...
1
vote
1answer
60 views

Most powerful test for deciding probability mass function

Let $X$ be an integer valued random variable supported on $\{0.1.2...,12\}$ whose pmf is either $g(x)=1/13; x=0,1,...,12$ or $ f(x)=\dfrac x {36} 1_{\{0,1,...,6\}} + (\dfrac 13 - \dfrac x{36})1_{\{7,...
2
votes
1answer
345 views

Comparing a distribution of percentages

I have two populations that I would like to compare, each of which contains N samples. Each sample itself contains many events that can be categorized into 20 different, mutually exclusive, categories....
0
votes
0answers
16 views

Statistical Test of Model Mean and Std.Dev

I have thousands of real-world observations for the input/output of some stochastic system. I have made a mathematical model that I have strong reason to believe governs the underlying relation ...
1
vote
0answers
22 views

Solution verification for a hypothesis testing question

I am posting this question as a solution checking. Let $X_1,...,X_{30}$ be a random sample from the exponential distribution with unknown mean $\mu\in \{1,1/\delta\}$ (where $\delta>1)$. Consider ...
0
votes
1answer
18 views

Why shift the mean of a bootstrap distribution when conducting a hypothesis test?

I'm wondering about why we do a particular thing (and not another) when conducting bootstrapped hypothesis tests. My understanding of bootstrapped hypothesis tests is this (based on this helpful ...
0
votes
1answer
48 views

'p-value' for clustering

I was just wondering if there is a way to check if the clustering found by a specific method (e.g., KMeans, DBSCAN, Mixture Models, ...) is significant. Something in line with, there is 5% chance that ...
0
votes
0answers
9 views

Comparing main effects and interactions on the same significance scale

When we collect genomics/metabolomics/proteomics data (many variables) in a between-by-within 2-way experimental design, we are often interested in which variables change differently between groups (...
1
vote
1answer
21 views

Are there statistical tests for testing multiple sample proportion?

For categorical variable testing, we have chi-square test. For example, we want to test if a dice is fair (assuming sample has [6,5,7,7,6,5] and ...
0
votes
2answers
351 views

Statistical t test for unequal sample sizes (paired samples)

I am analyzing the paired samples. Patients with drug at time point 0, at time point 12 weeks, time point 24 weeks. I would like to see the significant differences in the patients over the time ...
0
votes
1answer
19 views

Should I adjust for family-wise error when doing many logistic regressions and t tests

In my biomarker analysis, there are 100 biomarkers. Each biomarker is measured at 4 different time points. I'm looking for any biomarker that is associated with outcome variable at any timepoints. ...
18
votes
2answers
3k views

Proof that F-statistic follows F-distribution

In light of this question : Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom I would love to understand why $$ F = \frac{(\text{TSS}-\text{RSS})/(p-1)...
0
votes
2answers
25 views

Paired t-test with inequality in $H_0$

I have read that paired $t$-test works assuming a null hypothesis with equality, i.e: $$H_0 : \mu = \mu_0 $$ Is there any way to do the same test with an inequality, such as: $$H_0 : \mu \geq \mu_0 $$ ...
0
votes
1answer
15 views

How to measure risk of a Type 2 error in A/B tests

I'm conducting A/B tests where outcomes are binary (success or failure in control and experiment populations). The assessment of the risk of a Type 1 error is relatively straightforward: given an ...
6
votes
2answers
484 views

Two sample t-test to show equality of the two means

Given two (numeric) samples I would like to show that there is not a significant difference between the two means $\mu_{1}$ and $\mu_{2}$. If my goal was to show a significant difference I would ...
0
votes
1answer
365 views

How to check if a sample is representative of a population, across several criteria.

I have a total population of several hundred houses, which have different sizes, prices, and ages. I want to be able to select a random sample, of a few dozen or so houses, and test how ...
0
votes
2answers
34 views

different chi-square test results [duplicate]

I am doing chi-square tests using Scipy. Just realized the test statistics will change even though you simply rescale all data and the testing outcome could change. For example, using ...
2
votes
1answer
16 views

Analyse variation between distribution curves according to factors

I have a set of continuous distributions representing the leaf area density found at different heights through a forest canopy. e.g. like that found in Whitehurst et al. (2013): For each of these ...
0
votes
0answers
21 views

Statistical test to compare the mean of two samples

There's a competition with 20 categories, and each category has 3 winners. I have an array of the winner's performance of the next year (60 values), and other array with the average of the competitors ...
0
votes
0answers
6 views

Pre- and post-intervention difference between standard deviation scores

My first research question is based on establishing whether mental toughness is a trait like construct or a skill that can be manipulated through training and experience. I am using descriptive ...
1
vote
1answer
34 views

Bootstrap Hypothesis Test: Comparing the Performance of two Models

I have two models $A$ and $B$ as well as a training set $X$. I want to test whether there is a significant difference in performance between model $A$ and $B$. I'm attempting to do this via ...
2
votes
1answer
27 views

Testing for randomness against Poisson distribution - what is the null hypothesis?

"This week, we examined the simplest theory of spatial criminology – that crimes are distributed at random. When spatial criminologists want to test observed data for randomness, they test that data’s ...
10
votes
2answers
569 views

Why is the Neyman-Pearson lemma a lemma and not a theorem?

This is more of a history question than a technical question. Why is the ``Neyman-Pearson lemma'' a Lemma and not a Theorem? link to wiki: https://en.wikipedia.org/wiki/Neyman%E2%80%93Pearson_lemma ...
1
vote
0answers
16 views

Test for average of independent, non-Identical binomial distributions

I have a box with $m$ coins, each with a probability $p_i$ for $1\leq i \leq m$ of flipping heads. As an experiment, I flip each coin $n_i$ times, recording the results. What is a statistical test I ...
3
votes
0answers
38 views

P-value: Fisherian vs. contemporary frequentist definitions

I am trying to see if I understand the definition of $p$-value as used by Sir R. A. Fisher and the one used today by frequentist statisticians (not sure how to call it better). $p$-value according ...
3
votes
1answer
67 views

Permutation tests for sharp vs weak null hypothesis

Permutation tests provide an easy way to obtain well-calibrated tests for experimental (randomized) data for the sharp null of no treatment effect. The idea: In context where either only the ...
0
votes
0answers
18 views

How to choose confidence interval for clustered tests?

I'd like to test if requests to my system are succeeding at least 90% of the time. This system receives about 10 requests/minute. I'm ok getting a 1-in-20 false alarm. If my question ended here, I ...