Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

Filter by
Sorted by
Tagged with
3
votes
1answer
115 views

Why is the two-sample test giving me inconsistent results?

I am applying a two-sample t-test to determine whether we have software regressions on latency measurements. Procedure Run the test for build b1 and gather 60 latency measurements. Run the test for ...
1
vote
1answer
975 views

How to interptet Jensen Alpha statistical significance?

When you regress portfolio excess returns against relative benchmark excess return you get a model in which the beta (slope) could be interpreted as the one you get from the CAPM, that is systemic ...
0
votes
1answer
34 views

More than two periods' hypothesis testing

I want to test more than two periods' income from different goods. I want to know if my new sales strategy has had a significant impact on my income. Example: ...
2
votes
0answers
25 views

Multiple comparisons version of Kolmogorov-Smirnov test

Suppose $\mathbf{y}_{i} $ and $ \mathbf{x}_{i}\,$ are length-$m$ vectors and $D_{i}$ is some arbitrary distribution $(i=1,...,N)$. I would like to conduct the following hypothesis test: $ H_{0}: \...
6
votes
4answers
285 views

Defining extremeness of test statistic and defining $p$-value for a two-sided test

Our tag definition of the $p$-value says In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the ...
0
votes
1answer
33 views

How to compare conversion rates from AB test when users visit multiple times [duplicate]

I've been asked to compute the minimum sample size for an AB test that my company is conducting for a tweaked recommender system. We will choose two groups of users, one of which sees only the A ...
0
votes
0answers
12 views

GLR test for change in mean of Rician

I am trying to formulate the GLR test for the change in mean of a Rician distribution. I get the Rician pdf as $$ p(x) = \frac{x}{\sigma^2} \exp\left[ -\frac1{2\sigma^2}\left(x^2+\alpha^2\right) \...
3
votes
1answer
43 views

Two sample t test with equal variances hypothesis

Given a trial to test a new drug. There are 20 volunteers with some disease. The volunteers are split into two groups (10 volunteers in each) randomly: the treatment group and the control group. ...
1
vote
1answer
172 views

t-test in large, unequal sample size + additional info such as effect size

I have 2 samples, named "1" and "3": (1) n = 1282, mean = 14.77, sd = 8.27 (3) n = 5360, mean = 15.88, sd = 8.55 Their distribution look: ...
2
votes
1answer
797 views

Does Neyman-Pearson Lemma consider the case when the likelihood ratio equals the critical value?

Here are three different versions of Neyman-Pearson lemma. They differ in that the first two (books) ignore the case when the likelihood ratio equals the critical value, while the last one (Wikipedia) ...
1
vote
0answers
20 views

Cochran's Q but for more than two outcomes

Formal problem I have a random variable $X$ that takes (unordered) discrete values $i\in [0, 2]\subset\mathbb N$ (there is no order here, I just don't know how to write it neater) that is measured $t \...
1
vote
1answer
20 views

When to apply pvalues correction?

I have what I believe to be quite a beginner question. I'm testing the effect of a qualitative variable (2 groups) on 4 response variables. These 4 variables are correlated (see figure below), but I'm ...
1
vote
1answer
309 views

Likelihood ratio test always equal to $1$?

Consider a random sample of size $n$ from a discrete uniform distribution with pmf $f(x|N) = 1/N$, where $x=1,\dots,N$. Determine the likelihood ratio test for testing $H_0: N \leq N_0$, $H_1: N > ...
5
votes
0answers
326 views

A specific example of two-sided chi-squared test

First of all I have read the answers to this question, but I'm not happy with them, I feel that they miss the point that I'm willing to address here. I'm looking at a chi-squared test for ...
6
votes
1answer
2k views

How is pairwise PERMANOVA/adonis a valid non-parametric approach for pairwise comparisons

Assume that we have taken independent random samples of several individuals from 5 locations that represent 5 populations. The design is fairly unbalanced: the number of individuals sampled from each ...
0
votes
0answers
12 views

Demonstrating that proposed cutoff score is non optimal

I have a small data set with a test score and classification (positive or negative diagnosis). There is already clinical evidence that the cut point suggested by the test manufacturer produces too ...
2
votes
1answer
32 views

Understanding this hypothesis testing exercise using P-value

The exercise states that: Your team evaluates a new cannon ball and compares its distance performance to a new ball from your competitor. In an initial evaluation you test 10 of your new cannon balls ...
0
votes
0answers
8 views

Derive a formula for minimum sample size to achieve desired power for a 2 sample test, for fixed significance level

Suppose you have two populations, normally distributed with means $\mu_1$ and $\mu_2$, and standard deviations $\sigma_1$ and $\sigma_2$ respectively. Test whether they have the same mean. Suppose the ...
0
votes
1answer
39 views

Akaike‘s Information Criterion as a significance test

I stumbled upon a question related to AIC, where two nested models are given (no other information), and the difference between the number of parameters, $d$. The question states that the AIC can be ...
0
votes
2answers
75 views

How to prove statistical significance, python

I'm new in statistics and would be so grateful if you give me some insights: I have two big tables - results of work Model_1 and Model_2. I created and calculated statistics - something like Precision ...
0
votes
0answers
73 views

How to calculate Probabailty of (Null is true given Ho is rejected)?

Given: $\alpha = 0.05$, $\beta = 0.10$, $p = 0.5$ (Prior probabailty of $H_0$) How can we calculate $P(H_0 \text{ true} | H_0 \text{ rejected})$ Is it just $1 - \beta$? I was thinking that maybe ...
0
votes
0answers
22 views

Accepting vs failing to accept the null hypothesis? [duplicate]

In this paper the authors state the following regarding a null hypothesis significance test: If P-value < critical value, reject null hypothesis and accept the alternative hypothesis (difference ...
1
vote
0answers
45 views

How to know the number of subjects and events required for statistical significance? [duplicate]

Say I conduct a study where i measure physiological signal of a set of N subjects, to record a particular event. I am interested of the duration and the amplitude of that event. The study ends when I ...
3
votes
1answer
94 views

Chi Square Goodness of Fit vs. Igloo

I am very new to Chi square goodness of fit tests but have done a fair bit of research. Basically, I have the following 12 data points: 7392, 7656, 7241, 6164, 4984, 4664, 15262, 17053, ...
9
votes
1answer
168 views

Unit testing sampling methods

I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
2
votes
1answer
45 views

Test on a parameter in which $H_1$ is not complementary to $H_0$. Is this possible?

I have the following problem and I don't know how to solve it. We have a population of $N=20$ enterprises. Each enterprise has a probability $p$ to reduce his customers independently from the others. ...
2
votes
0answers
30 views

Measuring “uniqueness of fit” or “unique goodness of fit” of a model to some data

Given some parametric model, we can determine the best-fit parameters to some data set. Once we have quantified goodness of fit for each parameter set, we can easily generalize from this to measure ...
4
votes
1answer
65 views

Failing to obtain $\chi^2(1)$ asymptotic distribution under $H_0$ in a likelihood ratio test: example 1

I have a large sample (a vector) $\mathbf{x}$ from a random variable $X\sim N(\mu,\sigma^2)$. The variance $\sigma^2$ is known, but the expectation $\mu$ is unknown. I would like to test the null ...
2
votes
0answers
17 views

Testing the ratio of SLR coefficients

Consider applying simple linear regression (SLR) to the data $\{(X_{i},Y_{i})\}_{i=1}^{n}$. Furthermore, let's assume that the errors are conditionally normal, i.e $$Y_{i} = \beta_{0} + \beta_{1}X+e_{...
0
votes
0answers
37 views

What statistical test should be used for testing the association between ordinal data and 2 categorical variables?

I am trying to find the association between grades and most and least preferred subjects, wherein it is hypothesized that subject preference is the independent variable to the dependent variable ...
1
vote
1answer
28 views

How to test the relationship between variance of functional trait value (dependent variable) and elevation range of occurrence?

I have a data set consisting of functional trait values for trees across an elevation gradient. Hence each functional trait value (dependent variable) has a corresponding elevation value (independent ...
1
vote
1answer
41 views

Can I test this hypothesis with a one-tailed test?

A student wants to investigate whether people from northern USA are more severely affected by winter depression ("seasonal affective disorder") than people from southern USA, using a ...
3
votes
1answer
28 views

Performing hypothesis test with two categorical variables, where one variable has four ordered categories

I'm working with two categorical variables and need to test whether they're related. This isn't the exact problem I'm solving but the example should help illustrate the nature of the problem I'm ...
2
votes
1answer
54 views

Seeing if (and *how*) two categorical variables are related

Consider the variables sex (male or female) and political party affiliation (Democrat or Republican). In this case, the null hypothesis and alternative hypothesis would be: H0: Sex is independent ...
2
votes
1answer
3k views

How to detect intermittent time series?

I need to automatically identify if a time series is intermittent or not. Depending on the result I'll use one or another method for forecasting it. Is there any test to detect intermittent time ...
4
votes
2answers
85 views

Asymptotic null distribution of the LR statistic with point null and point alternative

I have a large sample (a vector) $\mathbf{x}$ from a random variable $X\sim N(\mu,\sigma^2)$. The variance $\sigma^2$ is known, but the expectation $\mu$ is unknown. I would like to test the null ...
2
votes
1answer
50 views

Why is a t statistic used for standardized regression coefficients?

I understand the sampling distribution of unstandardized linear regression coefficients is normal, and therefore a t distribution can be used to determine p values for given coefficient and standard ...
4
votes
4answers
238 views

Testing Hypothesis with different alternatives

I want to test whether $\mu=\mu_0$ where $\mu_0$ is some fixed number. Consider the following two different testings. Hypothesis Testing 1: $H_0:\mu=\mu_0,H_1:\mu<\mu_0$ Hypothesis Testing 2: $H_0:\...
0
votes
0answers
19 views

Which to Use: Likelihood Ratio Test or Uniformly Most Powerful Test?

I've recently been learning about MPTs (most powerful tests), UMPTs (uniformly most powerful tests) and LRTs (likelihood ratio tests), and do not totally understand in which context the different ...
2
votes
1answer
230 views

Adjust ROC analysis for multiple testing?

we did an exploratory prospective study in medicine in order to find parameters which are able to predict a specific post-surgical event (0/1) before the actual surgery. We have about 10 parameters ...
0
votes
0answers
28 views

How to test the difference between dependent partial correlations in the presence of autocorrelation?

Background and what I am trying to do: I have a single dependent variable $Y$, and $K$ independent variables of interest ($X_1, X_2... X_k$). I also have a single independent variable of non-interest, ...
1
vote
1answer
359 views

How to run Friedman test correctly across three sets of questionnaire data divided into sub-groups?

I am trying to evaluate the usability of three prototypes of an application and want to see if there is a significant difference in terms of usability. For this I have collected responses from users ...
0
votes
0answers
7 views

Why we use the number of params as degree of freedoom in the F test for regression?

In a linear regression, we can assume that $\frac{(SSD_y - SSR)/p-1}{SSR/(n-p-1)}$ have a $F$ distribution if the null hypothesis $Ho$ is true, with $SSR$ being the sum of residues of the regression ...
1
vote
2answers
31 views

Choosing a test for questionnaire analysis

I want to analyse the data from the following experiment setup: Participants were divided into three groups, each group read a story and had to anwser a question by choosing option A or B. The age, ...
0
votes
0answers
9 views

spatial correlation between two hexa/density maps

I was working on a report and was faced with a question regarding maps and spatial analysis. I want to compare some maps and find the similarities/differences among them. The maps represent the sum of ...
2
votes
1answer
54 views

What does it mean that the p-value is a function of the sample size?

I have heard the phrase the "p-value a function of the sample size" but am not sure if I really understand what it is saying. I normally refer to thinking about a t-test where the t-...
1
vote
0answers
21 views

Proper Test for Type of Data

Warning: I have not done any statistical analysis since 2012. Problem: I have a list of lists of lists: Each of the $289$ ($17\times 17$) element is $\{\phi_{j,1},\phi_{j,2},f(\phi_{j,1},\phi_{j,2}),\{...
0
votes
0answers
9 views

How to select variables for Little´s Test?

Lets assume there are 150 variables in a Dataset. For my analysis I only use around 50 of them. Of those 50 variables, there are 18 Variables which contain missing values. How should I perform the ...
1
vote
1answer
38 views

How to statistically test data points that are coming from two groups of distributions?

Suppose we have two groups of data points, i,e, A and B, and we want to know if they are coming from two different distributions or just one? The candidate solutions are standard t-test or Kolmogorov ...
0
votes
0answers
26 views

Correct test to verify that data stems from a (given) non-stationary categorical time-series

I have a categorical random variable that can take one of 3 values (red, blue, green). The variable is called Guess and - over the course of a time-continuous ...