Hypothesis testing assesses whether data support a given hypothesis rather than being an effect of random fluctuations or some other process described by an alternative hypothesis.
0
votes
0answers
15 views
Relation between observed power and p-value?
I am trying to understand the relation between observed power and p-value in Stephane's reply, which I think is based on J. M. Hoenig and D. M. Heisey (2001) "The Abuse of Power: The Abuse of Power ...
0
votes
0answers
6 views
Implications of lower-bounded total variation distance on hypothesis testing
Let $\{X_i\}_n$ be a sequence of $n$ random variables independently and identically drawn from either $P$ or $Q$. Thus the sequence $\{X_i\}_n$ has a product distribution, which is either $P^n$ or ...
0
votes
0answers
19 views
Significance level, effect size, power and sample size: why do any three determine the other?
In Statistical Power Analysis for the Behavioral Sciencies by Jack Cohen (from the limited preview on Google books), he said in a hypothesis testing, any three of significance level, effect size, ...
3
votes
0answers
29 views
How to test whether mean and variance is the same in two small samples?
I would like to test two relatively small samples against the null hypothesis that both their means and variances are the same. The alternative would be that they in fact differ. I saw a post on this ...
1
vote
0answers
19 views
How to extend multiple regression hypothesis tests to multivariate?
I have a basic multivariate regression model $\mathbf{Y}$ = $\mathbf{XB}$ + error
$\begin{bmatrix}y_{1i} \\ y_{2i} \\ \vdots \\ y_{ki} \end{bmatrix}$ =
$\begin{bmatrix} X_{11} & ... & X_{1p} ...
0
votes
0answers
20 views
Statistical Significance test for difference in FScore
For a classification task, I have developed two methods and FScore(harmonic mean of precision and recall) of both the classes serves as the performance criteria. How can I check whether the difference ...
2
votes
2answers
55 views
Why will a statistic be significant with sufficiently large samples unless the population effect is exactly zero?
From Wikipedia
Given a sufficiently large sample size, a statistical comparison will always show a significant difference unless the population effect size is exactly zero.
For example, a ...
2
votes
0answers
28 views
Does false discovery rate estimate some population quantity?
False discovery rate (FDR) is defined as
FDR = FP / (TP + FP).
Does it estimate some population quantity, independent of sample, when the sample satisfies some condition?
To make it clearer, ...
2
votes
1answer
39 views
What is the proper way of converting ordinal values to numbers?
I have two painkillers and I have given them to two groups and recorded how the pain level changed in three categories: "Helped a lot", "Slightly better", "Did not help"
Now I want to do a t-test ...
1
vote
1answer
23 views
Rejection regions nested or not?
When varying the significance level, the rejection regions can be chosen to be nested or not nested. I was wondering what some theoretical and practical considerations are in using either nested or ...
0
votes
0answers
16 views
Econometrics : White Test with R [migrated]
Good morning,
I am trying to realize the white test on my linear model with R. I don't know how to write the R codes to realize the White Test.
Price : house price, in millions dollars
Bdrms : ...
0
votes
1answer
25 views
Cointegration of a VAR(1) process
I am using a Johansen procedure to test for cointegration a vectorial 4-dim vector (timeserie).
First I tested for differential stationarity of each individual vector, all of those have a unit root ...
1
vote
0answers
15 views
About 2 unit root tests and null hypothesis
I have been looking at unit root testing. Specifically 2 tests:
The ADF test.
The ADF (augmented Dickey Fuller) test has the null hypothesis that "the time series has a unit root" (meaning that the ...
1
vote
0answers
14 views
Significance testing for a group of samples
For instance, I have a bar chart - in which I have 4 samples/bars. for these 4 bars, I want to check statistical significance for all combinations.
How can I do it? Can unpaired t-test be used - to ...
1
vote
3answers
58 views
How to test for simultaneous equality of choosen coefficients in logit or probit model?
How to test for simultaneous equality of choosen coefficients in logit or probit model ? What is the standard approach and what is the state of art approach ?
3
votes
2answers
58 views
Measuring the performance of Logistic Regression
Being quite new to the field, it occurs to me that there are multiple and fundamentally different ways of assessing the quality of a logistic regression:
One can evaluate it by looking at the ...
2
votes
0answers
32 views
Is the p-value still uniformly distributed when the null hypothesis is composite?
When the null hypothesis is simple (i.e., has only distribution of the sample) and the only distribution of the sample is continuous, the p-value can be shown to be uniformly distributed over (0,1).
...
1
vote
1answer
26 views
How to test a yes/no outcome with different inputs?
I have what I think is a very simple beginner question, but I do not have any formal training or knowledge of statistics or design of experiments.
Let's say I have a yes,no (0,1) outcome of an ...
0
votes
1answer
32 views
Working out probability with alpha and beta
Working out probability with alpha and beta?
Let's say in an experiment, the Null hypothesis is patients having Condition A. Alpha (type I error) is 0.05 and beta (type II error) is 0.15.
If i have ...
1
vote
0answers
53 views
How can statistical tests be categorized?
There are many tests in statistics. I noted that some of them may be grouped together, since they are used for similar purposes. For example
tests of independence
tests that compare distributions ...
2
votes
0answers
31 views
How to compare two matrices?
I am working on Markov transition matrices. I would like to find a statistical test to compare them.
The first matrix is considered the population transition matrix and the second one is obtained by ...
1
vote
3answers
47 views
Testing whether two groups formed from a continuous variable differ on a binary outcome
18 people, who have been divided into 2 groups based on the semiquantitative expression of a protein (low vs high).
Group 1 (low protein levels) has 6 people, Group 2 (high protein levels) has 12 ...
3
votes
2answers
103 views
+50
Exponential family in testing and estimation
In the Annals of Statistics paper "Defining the curvature of a statistical problem(with applications to second order efficiency)" by Bradley Efron, he claims the following two statements in the first ...
1
vote
1answer
53 views
How to test if two samples are distributed from the same Gaussian process
Given a sequence $\mathbf{x} = (x_1,x_2,\dots,x_n)$ which is sampled from some Gaussian process $GP(\mu_1,\Sigma_1)$ and a "target" sequence $\mathbf{y} = (y_1,y_2,\dots,y_n)$ sampled from another ...
0
votes
0answers
10 views
Underlying physical basis of an exponential distribution [migrated]
My data set of upper atmospheric cloud occurrences N versus their thickness (or optical brightness, say B) show an exponential variation over more than two orders of magnitude, that is N varies as ...
2
votes
0answers
13 views
How to test significance of pre-post difference scores on 8 measures (1 group)
I have a set of pre and post treatment scores on 8 measures. I'd like to test which of these show significant improvement after treatment. If I use within-subjects t tests it will mean carrying out 8 ...
0
votes
0answers
30 views
Hypothesis testing for normal distribution
This question seems easy but I am not getting it. Let $X_{1},X_{2}, · · · ,X_{15}$ be a random sample from a normal population
$X ∼ N(0, σ^2)$. Find a best critical region of size $α = 0.05$ for ...
2
votes
1answer
67 views
How to compare two datasets using metrics drawn from unknown distributions and with small sample sizes?
I have two datasets consisting of metrics from several experiments. Dataset 1 is the collection of results of experiments E performed by user A on product A, repeated N times. Dataset 2 is the ...
0
votes
0answers
47 views
Testing hypothesis
This question may be very easy and not good to ask here may be but I am just learning the testing hypothesis. I have no idea how to do it. Let $X_1,X_2,\dots,X_n$ be a random sample from an ...
6
votes
4answers
104 views
Statistical Tests That Incorporate Measurement Uncertainty
Suppose I am given two groups of mass measurements (in mg), which are referred to as y1 and y2. I want to do a test to determine if the two samples are drawn from populations with different means. ...
0
votes
1answer
54 views
What method can be used to test if three or more categorical sample data sets are from the same distribution?
I have three data sets like this:
data1: {A, A, B, C, D, ..}
data2: {A, B, B, C, E, ...}
data3: {A, C, D, D, E, ...}
How do I test if these three data sets are from the same distribution?
3
votes
0answers
146 views
Ratios of means - statistical comparison test using Fieller's theorem?
I would really appreciate any suggestions with the following data analysis issue. Please read till the end as the problem at first may appear trivial, but after much researching, I assure you it is ...
1
vote
0answers
10 views
Conditions for the existence of UMP in one-sided two sample hypothesis test
Assume that there are two samples from, possibly, two distinct distributions.
I would like to find a UMP test $H_0: p_1 = p_2 = x$ versus $H_1: x < p_1 < p_2$. If that is not possible, I would ...
3
votes
0answers
64 views
Sufficient statistic and hypothesis testing
Suppose I have a family of (continuous) distributions $\mathcal{P}=\{P_\theta(x),\theta\in\mathbb{R}^+\}$. I also have a statistic $T(x)$ that is sufficient for $\theta$.
The value of the parameter ...
1
vote
0answers
28 views
Neyman-Pearson Theorem Question
I have the following question for my homework:
Suppose X~exp($\theta$).
We want to test $H_0: \theta=1 vs. H_a:\theta=2$, based on a sample of size 2 - ${X_1,X_2}.$
a. Obtain the most powerful test ...
1
vote
1answer
45 views
Using continuity correction for normal approximation or not?
Below is a question on a recent actuarial exam, Exam 3L of the CAS. I didn't know whether or not to use the continuity correction when using the normal approximation to do hypothesis testing ...
3
votes
1answer
58 views
Are the posteriors “different”? How does one discuss the result?
First, I realize this may be a basic question. However, when I search the web for references on this issue, I run into the problem of wondering if the description I'm reading is applicable to the ...
1
vote
0answers
19 views
Test regression parameter against a constant in SPSS
This is a pretty basic question, but I can't find an answer by searching for different statements of the same problem.
Is there a straightforward way to test if a regression parameter is different ...
1
vote
0answers
21 views
What is a similar test?
From Wald. Likelihood ratio, and lagrange multiplier tests in econometrics by Robert f. Engle:
(Tests whose) size does not depend upon the particular value of $\theta \in$ null $\Theta_0$ are ...
0
votes
0answers
42 views
STATA - Procedure for properly estimating an AR(p)
I'm trying to estimate an autoregressive process AR(p).
Following the literature:
1) I checked if the series is stationary or not running the augmented Dickey-Fuller test (as I expected, the ...
1
vote
1answer
52 views
Forecasting optimization techniques in fantasy baseball
I am currently trying to build a better forecasting model for my fantasy baseball roster. I currently am using commonly accepted projected season statistics (ZiPS from Fangraphs) to determine the ...
0
votes
0answers
25 views
Statistics - hypothesis testing
(1) (3 points each) John has come up with a unique new remedy for insomnia. He knows that in the absence of his remedy insomniacs sleep an average of 250 minutes a night with a standard deviation of ...
2
votes
0answers
39 views
Is the p-postulate true?
The p-postulate is the notion that equal p-values provide equal evidence against the null hypothesis.
Royall, R. N. (1986). The Effect of Sample Size on the Meaning of Significance Tests, The ...
3
votes
3answers
90 views
Appropriate statistical test to test if probabilities are accurate
I have some data that looks like this:
Prob Outcome
0.09 0
0.10 0
0.10 0
0.11 1
0.84 1
0.99 1
0.86 1
0.78 1
0.86 1
0.00 0
etc.
...
3
votes
1answer
50 views
Test for differences between parameters of models estimated from partially overlaping samples
Suppose I have $n$ observations $(\pmb{x}_i,y_i)$ and two subsets of indexes of $\{1:n\}$, $S_1$ and $S_2$, with $S_1\neq S_2$, $\#\{S_1\}\neq\#\{S_2\}$, and $\{S_1\cap S_2\}\neq\emptyset$.
Suppose I ...
3
votes
1answer
31 views
How to prove the number of distinct distributions in a group of distributions?
Let's say we had 5 distributions: A,B,C,D,E.
An ANOVA test would tell us whether or not all of the means are equal, and thus a low p-value would mean at least one of the means is unlikely to be ...
1
vote
1answer
29 views
Is McNemar appropriate for this pre-post test design?
I recently came across a paper that uses the McNemar test to evaluate the effectiveness of an intervention to improve adherence to treatment. The study used a pre/post test design whereby the ...
1
vote
1answer
38 views
Two forms of test statistics in likelihood ratio tests
From Bickel and Doksum's Mathematical Statistics
I was wondering what "$\Theta_0$ is of smaller dimension than $\Theta = \Theta_0 \cup \Theta_1$" means?
Note: In the original text, "$\theta_0$" ...
0
votes
1answer
49 views
Does Neyman-Pearson Lemma consider the case when the likelihood ratio equals the critical value?
Here are three different versions of Neyman-Pearson lemma. They differ in that the first two (books) ignore the case when the likelihood ratio equals the critical value, while the last one (Wikipedia) ...
0
votes
1answer
76 views
KPSS test - output interpretation in stata
I did KPSS test for some variables in stata to check for stationarity; I want to interpret the the stata outputs, but I don't know how to do that. For instance, in the following case:
...
