Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

Filter by
Sorted by
Tagged with
2
votes
4answers
1k views

What's the appropriate spss test?

Context I have 25 men and 25 women as participants, and they did exactly the same thing: Each of them heard an attractive dialogue and they had to choose between a photo of a woman in red and a woman ...
2
votes
0answers
114 views

Comparing fatality rates

I have case fatality rates (deaths per 100 cases) for 2 different states receiving different treatments for 17 years. What is the best statistical method to compare them? Relative risk, odds ratio, ...
9
votes
1answer
168 views

Unit testing sampling methods

I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
6
votes
1answer
885 views

Is there a test of monotony, convexity or concavity?

I have a problem that I don't think I've met before. I have N observations of the variables v1 and v2 and I assume that there is a function f such as v2 = f(v1). I want to know if f has a particular '...
4
votes
1answer
227 views

Multiple simulations of a system under different conditions - paired data?

I am currently generating data by simulating a model of chemical system under different conditions (temperature) over time. In each simulation, the starting structure being modeled is exactly the same ...
0
votes
2answers
723 views

What test do I use / how do I show in the analysis that I measured everyone twice?

I have already asked a question on this forum about this, but as I changed my experiment a bit and I am still confused. I conducted an experiment in which 25 men and 25 women listened to an ...
0
votes
1answer
226 views

Appropriate test for testing a pair of random binomial variables

I have pairs of values from two runs (replicate) for each sample along with total count for each run. I assumed each value as random binomial. I used log-likelihood ratio test to compare each pair, ...
3
votes
2answers
861 views

Difference between Hausdorff and earth mover (EMD) distance

I have two dataset that i want to compare. each dataset contain the weight of 10 different person measured for 3 different day. I am interested in measuring the probabily that the two sample ...
9
votes
1answer
1k views

Comparing backtesting returns with real trading returns

I have 10 years of backtested simulated performance of some trading strategy (using historical prices), and N months of actual trading performance. What statistical test can I do find out if I'm on ...
5
votes
1answer
172 views

Sample size for actual vs backtested performance

I have 10 years of backtested simulated performance of some trading strategy (using historical prices), and N months of actual trading performance. I want to compare the two. How big does N have to be ...
2
votes
1answer
2k views

R code for two sample t-test in case of equal variances

In R, how can I perform a two sample t-test in the case of equal variances? My actual problem is to compare different algorithms for length measurements in machine vision. I have studied Applied ...
16
votes
4answers
22k views

How to specify the null hypothesis in hypothesis testing

What is a good rule of thumb for how to choose the question for the null hypothesis. For instance, if I want to check if the hypothesis B is true, should I use B as the null, B as the alternative ...
1
vote
0answers
117 views

How to know whether the six spatial features values (in 3-time period) significantly differ from each other? [closed]

I am currently bothered on how to statistically analyze my simple data as shown below. ...
23
votes
4answers
11k views

Testing significance of peaks in spectral density

We sometimes use spectral density plot to analyze periodicity in time series. Normally we analyze the plot by visual inspection and then try to draw a conclusion about the periodicity. But have the ...
2
votes
1answer
6k views

What is a null conjunction analysis in an fMRI study?

I'm reading an article, The commonality of neural networks for verbal and visual short-term memory (Majerus et al., J Cogn Neurosci 2010 22(11): 2570), about brain imaging in which the results are ...
3
votes
2answers
5k views

Testing the significance of differences between ratios with small samples

This is for a friend of mine. As an econometrician used to rely on large samples for inference, I find myself unsure whether the answer I have in mind is the best. Suppose we have four continuous ...
1
vote
1answer
205 views

How to test whether the majority of vertices belong to one giant component?

I have a sample of 200 independent networks and I want to test the hypothesis that majority of vertices belong to one giant component. I wonder what is the appropriate approach to do that. More ...
3
votes
2answers
2k views

Analyzing multiple choice survey questions

Consider the following survey question: Q: Choose one or more of the following 5 items: A B C D E How can one test which items are more frequently chosen for a sample with 100 individuals? Is it ...
5
votes
3answers
2k views

Help with data analysis of small datasets

I have the following dataset (triplicate values of 5 independent measurements and duplicate values of a control): ...
24
votes
3answers
17k views

Sanity check: how low can a p-value go?

I am using a ranksum test to compare the median of two samples ($n=120000$) and have found that they are significantly different with: p = 1.12E-207. Should I be ...
6
votes
1answer
319 views

Testing the importance of an item among a finite set of items

Consider the following survey question: Q: How would you classify the importance for you of the following 5 items: A B C D E Assign to each item a number in the ...
5
votes
1answer
5k views

Comparing means across two groups and over four time points when group sample sizes are very small

I have two groups (experimental, N=6, and control group, N=20). For each participant I measured a score (let say mean reaction time) 4 times. I would like to check: whether these groups differed in ...
3
votes
3answers
324 views

How to check hypothesis about estimation of random variable with unknown distribution?

I have one data sample of non-negative random variable $X$ with unknown distribution and predefined expected value $y$. Is there any test able to check null hypothesis $\mathbb{E}[X]\geq y$ or $\...
11
votes
2answers
3k views

What is the power of the regression F test?

The classical F-test for subsets of variables in multilinear regression has the form $$ F = \frac{(\mbox{SSE}(R) - \mbox{SSE}(B))/(df_R - df_B)}{\mbox{SSE}(B)/df_B}, $$ where $\mbox{SSE}(R)$ is the ...
6
votes
5answers
1k views

Textbook with list of hypothesis tests and practical guidance on use

I found a list of statistical tests along with practical guidance on Wikipedia. Can anyone point me to something similar, but in a textbook form? I'm interested in particular in practical guidance (...
3
votes
1answer
384 views

Tests on binomial distribution

I have the observations $X(n)$, where $X(n)$ is the realization of a binomial random variable with probability of success $p(n)$, and with $Y(n)$ trials. The observations are independent across $n$. ...
6
votes
1answer
1k views

How to tell the “closeness” of two variables

I have completed an experiment measuring the magnetic field of a solenoid. The answer I got was $0.0075 \pm 0.0011$T. The uncertainty by the way was generated from Microsoft Excel's Regression under ...
54
votes
3answers
2k views

Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
3
votes
2answers
892 views

Visual representations of the p-value in ANOVA to assist intuitive understanding

I'm a statistics newbie and I would like to make sense of the p-value with ANOVA, hopefully with a visual presentation. All online visual tools I've found so far only have MSB, MSE and F values ...
5
votes
2answers
10k views

Critical values for Anderson-Darling test

I've found critical values for the Anderson Darling test for a Normal Distribution at 1%, 2.5%, 5%, 10% and 15% significance levels from various sources, including wikipedia: http://en.wikipedia.org/...
4
votes
3answers
1k views

Testing difference between two means with pairwise data and absence of normality

I am trying to compare the difference between two means with two pairwise samples. Unfortunately, my data are very far of being normal. What test would you recommend to use in this situation? Should I ...
2
votes
0answers
113 views

R: statistical test

I am working on 4 different species of tomatoes. From the data I had, I looked at the occurrence of a particular "event" in certain intervals of their genome (this interval is identical in all 4 ...
1
vote
3answers
290 views

Determining causality from a natural experiment

Suppose I thought that ingesting greater than 100 mg of chemical X annually noticeably decreased one's weight. Also, I had data (from a "natural" experiment) from 100 people (some male and some female)...
1
vote
2answers
156 views

Probability that X<=Y when X and Y are p values from two different hypotheses

If i test two hypotheses, one of which has a null that is -in fact- false and the other has a null that is -in fact- true, I want to know the probability that the first test will obtain a p value less ...
4
votes
2answers
399 views

What to do with a small (27) medical dataset?

I'm working with a lot of data that was collected by obstetricians regarding the health of infants (birth weight, gestational age at delivery, mother's BMI), and I am trying to connect this data with ...
2
votes
2answers
233 views

How to properly analyze distance from a reference?

I'm measuring distances of various samples from a reference point. The distance is defined as a non-negative number, where $d=0$ means that the test case is identical to the reference. My general ...
66
votes
3answers
5k views

References containing arguments against null hypothesis significance testing?

In the last few years I've read a number of papers arguing against the use of null hypothesis significance testing in science, but didn't think to keep a persistent list. A colleague recently asked me ...
8
votes
1answer
2k views

How to analyse repeated measure ANOVA with three or more conditions presented in randomised order?

Context: My question concerns a typical design in my area – a researcher takes a group of subjects (say 10) and then applies three different conditions to them to measure the change in a response ...
6
votes
1answer
9k views

Friedman's test and post-hoc analysis

I am carrying out a statistical analysis for my research. I am using a Friedman's test with post hoc analysis. At present I am using the function ...
3
votes
1answer
1k views

Likelihood ratio test vs. $\chi^2$/Z-test for comparing binomial datasets

When does/can one use the likelihood ratio significance test instead of Fisher's exact test or its Pearson $\chi^2$ approximation for comparing two binomial datasets? Given two binomial datasets (...
3
votes
1answer
574 views

How to calculate sample size for a one-sided test on a rxs contigency table

I work quite often with Google's Website Optimizer, essentially it allows you to make small changes to a site to determine if they have an effect on the ratio of clicks to a web page to sales. The ...
2
votes
1answer
211 views

How different random number generators can be more similar than identical ones?

I have 6 random number generators. They are "black boxes", i.e. I do not know if they are the same or different. For example I do not know if they provide the same arithmetic averages and/or root mean ...
1
vote
0answers
105 views

Testing a model with truncated data

This is probably a simple question. I'm studying events which have N outcomes, of which exactly one is correct. N is very large, more than a billion (and is known). There are many possible events, ...
19
votes
5answers
7k views

Changing null hypothesis in linear regression

I have some data that is highly correlated. If I run a linear regression I get a regression line with a slope close to one (= 0.93). What I'd like to do is test if this slope is significantly ...
1
vote
1answer
131 views

How to compare vitamin D and glucose levels between patients and controls?

I want to test the hypothesis of a decreased level of vitamin D in diabetic subjects. For this I have recorded blood glucose and vitamin D levels in 40 cases and 40 controls. What kind of ...
5
votes
1answer
1k views

What is explosive variable?

I'm trying to understand the following claim: if the $t$-statistic is greater than zero, it indicates that the variable is explosive... but does that mean it has unit root? In the context of ...
95
votes
3answers
16k views

What are examples where a “naive bootstrap” fails?

Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of ...
23
votes
2answers
43k views

Can a small sample size cause type 1 error?

I've learnt that small sample size may lead to insufficient power and type 2 error. However, I have the feeling that small samples just may be generally unreliable and may lead to any kind of result ...
5
votes
1answer
2k views

Geometric mean and standard deviation

standard deviation (std) can be used when arithmetic mean is the right statistic. What is the equivalent of std when geometric mean is the right statistic?
14
votes
5answers
28k views

Can chi square be used to compare proportions?

I've read that the chi square test is useful to see if a sample is significantly different from a set of expected values. For example, here is a table of results of a survey regarding people's ...