Hypothesis testing assesses whether data support a given hypothesis rather than being an effect of random fluctuations or some other process described by an alternative hypothesis.
0
votes
0answers
5 views
Why is controlling FDR less stringent than controlling FWER?
I have read that controlling FDR is less stringent than controlling FWER, such as in Wikipedia:
FDR controlling procedures exert a less stringent control over false discovery compared to ...
0
votes
0answers
16 views
Which variable vary previously to the other one
Here are two variable (a and b) and their value through time (time.a, time.b)
...
1
vote
0answers
16 views
Mallows distance hypothesis test: p-value =?
I am using Mallow's Distance (normalized Earth Mover's Distance) to characterize the similarity between two histograms. This is working very well, but I would like to identify a specific cut-off where ...
2
votes
2answers
45 views
How to choose the test statistic for permutation test?
Is the purpose of permutation test to test the null that several groups of samples come from the same distribution?
I found its steps are
The steps in a permutation-based computation of the ...
1
vote
1answer
39 views
Confusion about P-values
Okay, this might seem like a stupid question and it sure feels that way. And I have probably just lost myself by overthinking it or something, but here goes:
In Cohrans C-test you take the ratio ...
0
votes
2answers
71 views
When does replication reveal fraud?
Found this question here, and I couldn't answser it and thought it would be a wonderful question for this site.
Here's a little thought experiment for your weekend pleasure. Consider the ...
1
vote
0answers
29 views
How do I analyse a two-alternative forced choice test?
I have recently as part of my bachelor project at my university conducted a two-alternative forced choice test. I have looked at ways to say something with significance about which images (the test ...
7
votes
1answer
58 views
Is there a general definition of the effect size?
The effect-size tag has no wiki. The wikipedia page about the effect size does not provide a precise general definition. And I have never seen a general definition ...
2
votes
1answer
58 views
F test for equality of variances
I know that the test statistic is $$F=S_1^2/S_2^2 $$
But I am looking at some example questions from my lecturer and some have confused me. For example:
For a certain game, individual game scores ...
1
vote
0answers
29 views
Likelihood ratio test for a random variable following the Gamma Distribution
Assuming we have a random variable $X\sim \operatorname{Gamma}\left(\alpha,\beta \right )$:$\frac{1}{\Gamma (\alpha )\beta ^{\alpha}}x^{\alpha-1}e^{\frac{-x}{\beta }}$
I'd like to test the ...
4
votes
1answer
66 views
Ways of choosing test statistics: likelihood-based or not?
I just learned that there are three ways of constructing test statistics based on likelihood: likelihood ratio test, Wald test, and score test. I think it nicely categorizes various tests, and I liked ...
10
votes
2answers
88 views
Can Neyman-Pearson lemma apply to the case when simple null and alternative don't belong to the same family of distributions?
Can the Neyman-Pearson lemma apply to the case when a simple null and
a simple alternative don't belong to the same family of distributions?
From its proof, I don't see why it can't.
For example, ...
0
votes
0answers
16 views
Idea of the Nyblom test?
The nyblom-hansen test gives informations about the stability of the estimated parameters in a model. As far as I understand this test, it looks at the score of the ML at evaluates, how near to zero ...
3
votes
1answer
77 views
Relation between observed power and p-value?
I am trying to understand the relation between observed power and p-value in Stephane's reply, which I think is based on J. M. Hoenig and D. M. Heisey (2001) "The Abuse of Power: The Abuse of Power ...
0
votes
0answers
12 views
Implications of lower-bounded total variation distance on hypothesis testing
Let $\{X_i\}_n$ be a sequence of $n$ random variables independently and identically drawn from either $P$ or $Q$. Thus the sequence $\{X_i\}_n$ has a product distribution, which is either $P^n$ or ...
0
votes
0answers
26 views
Significance level, effect size, power and sample size: why do any three determine the other?
In Statistical Power Analysis for the Behavioral Sciencies by Jack Cohen (from the limited preview on Google books), he said in a hypothesis testing, any three of significance level, effect size, ...
4
votes
0answers
37 views
How to test whether mean and variance is the same in two small samples?
I would like to test two relatively small samples against the null hypothesis that both their means and variances are the same. The alternative would be that they in fact differ. I saw a post on this ...
1
vote
0answers
23 views
How to extend multiple regression hypothesis tests to multivariate?
I have a basic multivariate regression model $\mathbf{Y}$ = $\mathbf{XB}$ + error
$\begin{bmatrix}y_{1i} \\ y_{2i} \\ \vdots \\ y_{ki} \end{bmatrix}$ =
$\begin{bmatrix} X_{11} & ... & X_{1p} ...
0
votes
0answers
25 views
Statistical Significance test for difference in FScore
For a classification task, I have developed two methods and FScore(harmonic mean of precision and recall) of both the classes serves as the performance criteria. How can I check whether the difference ...
3
votes
2answers
81 views
Why will a statistic be significant with sufficiently large samples unless the population effect is exactly zero?
From Wikipedia
Given a sufficiently large sample size, a statistical comparison will always show a significant difference unless the population effect size is exactly zero.
For example, a ...
2
votes
1answer
40 views
What is the proper way of converting ordinal values to numbers?
I have two painkillers and I have given them to two groups and recorded how the pain level changed in three categories: "Helped a lot", "Slightly better", "Did not help"
Now I want to do a t-test ...
1
vote
1answer
27 views
Rejection regions nested or not?
When varying the significance level, the rejection regions can be chosen to be nested or not nested. I was wondering what some theoretical and practical considerations are in using either nested or ...
0
votes
0answers
16 views
Econometrics : White Test with R [migrated]
Good morning,
I am trying to realize the white test on my linear model with R. I don't know how to write the R codes to realize the White Test.
Price : house price, in millions dollars
Bdrms : ...
0
votes
1answer
27 views
Cointegration of a VAR(1) process
I am using a Johansen procedure to test for cointegration a vectorial 4-dim vector (timeserie).
First I tested for differential stationarity of each individual vector, all of those have a unit root ...
1
vote
0answers
18 views
About 2 unit root tests and null hypothesis
I have been looking at unit root testing. Specifically 2 tests:
The ADF test.
The ADF (augmented Dickey Fuller) test has the null hypothesis that "the time series has a unit root" (meaning that the ...
1
vote
0answers
15 views
Significance testing for a group of samples
For instance, I have a bar chart - in which I have 4 samples/bars. for these 4 bars, I want to check statistical significance for all combinations.
How can I do it? Can unpaired t-test be used - to ...
1
vote
3answers
67 views
How to test for simultaneous equality of choosen coefficients in logit or probit model?
How to test for simultaneous equality of choosen coefficients in logit or probit model ? What is the standard approach and what is the state of art approach ?
4
votes
2answers
62 views
Measuring the performance of Logistic Regression
Being quite new to the field, it occurs to me that there are multiple and fundamentally different ways of assessing the quality of a logistic regression:
One can evaluate it by looking at the ...
2
votes
0answers
34 views
Is the p-value still uniformly distributed when the null hypothesis is composite?
When the null hypothesis is simple (i.e., has only distribution of the sample) and the only distribution of the sample is continuous, the p-value can be shown to be uniformly distributed over (0,1).
...
1
vote
1answer
27 views
How to test a yes/no outcome with different inputs?
I have what I think is a very simple beginner question, but I do not have any formal training or knowledge of statistics or design of experiments.
Let's say I have a yes,no (0,1) outcome of an ...
0
votes
1answer
33 views
Working out probability with alpha and beta
Working out probability with alpha and beta?
Let's say in an experiment, the Null hypothesis is patients having Condition A. Alpha (type I error) is 0.05 and beta (type II error) is 0.15.
If i have ...
1
vote
0answers
53 views
How can statistical tests be categorized?
There are many tests in statistics. I noted that some of them may be grouped together, since they are used for similar purposes. For example
tests of independence
tests that compare distributions ...
2
votes
0answers
32 views
How to compare two matrices?
I am working on Markov transition matrices. I would like to find a statistical test to compare them.
The first matrix is considered the population transition matrix and the second one is obtained by ...
1
vote
3answers
48 views
Testing whether two groups formed from a continuous variable differ on a binary outcome
18 people, who have been divided into 2 groups based on the semiquantitative expression of a protein (low vs high).
Group 1 (low protein levels) has 6 people, Group 2 (high protein levels) has 12 ...
3
votes
2answers
130 views
Exponential family in testing and estimation
In the Annals of Statistics paper "Defining the curvature of a statistical problem(with applications to second order efficiency)" by Bradley Efron, he claims the following two statements in the first ...
1
vote
1answer
53 views
How to test if two samples are distributed from the same Gaussian process
Given a sequence $\mathbf{x} = (x_1,x_2,\dots,x_n)$ which is sampled from some Gaussian process $GP(\mu_1,\Sigma_1)$ and a "target" sequence $\mathbf{y} = (y_1,y_2,\dots,y_n)$ sampled from another ...
0
votes
0answers
11 views
Underlying physical basis of an exponential distribution [migrated]
My data set of upper atmospheric cloud occurrences N versus their thickness (or optical brightness, say B) show an exponential variation over more than two orders of magnitude, that is N varies as ...
2
votes
0answers
15 views
How to test significance of pre-post difference scores on 8 measures (1 group)
I have a set of pre and post treatment scores on 8 measures. I'd like to test which of these show significant improvement after treatment. If I use within-subjects t tests it will mean carrying out 8 ...
0
votes
0answers
30 views
Hypothesis testing for normal distribution
This question seems easy but I am not getting it. Let $X_{1},X_{2}, · · · ,X_{15}$ be a random sample from a normal population
$X ∼ N(0, σ^2)$. Find a best critical region of size $α = 0.05$ for ...
2
votes
1answer
69 views
How to compare two datasets using metrics drawn from unknown distributions and with small sample sizes?
I have two datasets consisting of metrics from several experiments. Dataset 1 is the collection of results of experiments E performed by user A on product A, repeated N times. Dataset 2 is the ...
0
votes
0answers
47 views
Testing hypothesis
This question may be very easy and not good to ask here may be but I am just learning the testing hypothesis. I have no idea how to do it. Let $X_1,X_2,\dots,X_n$ be a random sample from an ...
6
votes
4answers
110 views
Statistical Tests That Incorporate Measurement Uncertainty
Suppose I am given two groups of mass measurements (in mg), which are referred to as y1 and y2. I want to do a test to determine if the two samples are drawn from populations with different means. ...
0
votes
1answer
54 views
What method can be used to test if three or more categorical sample data sets are from the same distribution?
I have three data sets like this:
data1: {A, A, B, C, D, ..}
data2: {A, B, B, C, E, ...}
data3: {A, C, D, D, E, ...}
How do I test if these three data sets are from the same distribution?
3
votes
0answers
157 views
Ratios of means - statistical comparison test using Fieller's theorem?
I would really appreciate any suggestions with the following data analysis issue. Please read till the end as the problem at first may appear trivial, but after much researching, I assure you it is ...
1
vote
0answers
10 views
Conditions for the existence of UMP in one-sided two sample hypothesis test
Assume that there are two samples from, possibly, two distinct distributions.
I would like to find a UMP test $H_0: p_1 = p_2 = x$ versus $H_1: x < p_1 < p_2$. If that is not possible, I would ...
3
votes
0answers
66 views
Sufficient statistic and hypothesis testing
Suppose I have a family of (continuous) distributions $\mathcal{P}=\{P_\theta(x),\theta\in\mathbb{R}^+\}$. I also have a statistic $T(x)$ that is sufficient for $\theta$.
The value of the parameter ...
1
vote
0answers
28 views
Neyman-Pearson Theorem Question
I have the following question for my homework:
Suppose X~exp($\theta$).
We want to test $H_0: \theta=1 vs. H_a:\theta=2$, based on a sample of size 2 - ${X_1,X_2}.$
a. Obtain the most powerful test ...
1
vote
1answer
50 views
Using continuity correction for normal approximation or not?
Below is a question on a recent actuarial exam, Exam 3L of the CAS. I didn't know whether or not to use the continuity correction when using the normal approximation to do hypothesis testing ...
3
votes
1answer
59 views
Are the posteriors “different”? How does one discuss the result?
First, I realize this may be a basic question. However, when I search the web for references on this issue, I run into the problem of wondering if the description I'm reading is applicable to the ...
1
vote
0answers
23 views
Test regression parameter against a constant in SPSS
This is a pretty basic question, but I can't find an answer by searching for different statements of the same problem.
Is there a straightforward way to test if a regression parameter is different ...
