Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

4
votes
1answer
125 views

Construct probability that measurement belongs to one of two sets

We are dealing with a measurement apparatus that is cursed by noise and are trying to find out if a measurement was noise or an actual measurement. Assume we have a beam of light incident on an ...
3
votes
1answer
416 views

Moving from conversion rates to sales volume in A/B tests

there is a short introduction to AB Tests in this question or here at 20bits. We are currently testing different versions of landing pages and are using the conversion rate (e.g. 4% vs. 5%) to track ...
5
votes
2answers
7k views

How to estimate a critical value of Spearman's correlation for n=100?

I have searched a lot, and I can only find tables that show critical values up to n=30. Can someone provide, or point me to, a simple method of estimating this value for different $\alpha$?
5
votes
1answer
3k views

Two sample t-test for data (maybe time series) with autocorrelation?

I am new to statistics, so pardon any mistakes in my question. I have two time series $X_i$ and $Y_i$. Assuming that they're stationary AR(1) processes with possibly different means, how do I test ...
6
votes
2answers
4k views

Which non-parametric test can I use to identify significant interactions of independent variables?

I have an experiment producing results (dependent variables) that don't pass tests of normality, thus I am testing hypotheses using non-parametric tests. My DVs are continuous, while my factors (...
14
votes
1answer
17k views

Is there any difference between the terms “paired t-test” and “pairwise t-test”?

What is paired t-test, and under which circumstances should I use paired t-test? Is there any difference between paired t-test and pairwise t-test?
3
votes
1answer
220 views

Appropriate test to compare online voting data

The data I want to analyze consist of set of votes similar to the voting system here on stackexchange. Votes are binary, i.e, items can receive up- or down-votes. The data have been gathered in an A/B ...
4
votes
2answers
795 views

Difference between two slopes

I have a dataset with yearly levels of corruption in a number of countries, as well as whether they changed their government that year. ...
7
votes
1answer
205 views

Location test under a bounded non-stationarity?

Suppose I draw $n$ observations $X_1,X_2,\ldots,X_n$ independently from a distribution where $X_i \sim \mathcal{N}(\mu_i,\sigma^2)$, where the mean is assumed to be Lipschitz: $\left| \mu_i - \mu_{i+...
2
votes
4answers
3k views

Justifying normal approximation of experimental data

Background: In connection with the question here I came upon a more interesting question. I believe the question is large and distinct enough to have it's own thread. Of course I might be mistaken, ...
5
votes
2answers
547 views

What is the right statistical test to use when attesting claims like “60% people switched over from group A to group B”?

My problem is this: There are 52 people divided into 5 groups: A, B, C, D, and E. From stage 1 to stage 2, they either stay in the same groups, or change groups. The following table shows how many ...
56
votes
8answers
6k views

Why continue to teach and use hypothesis testing (when confidence intervals are available)?

Why continue to teach and use hypothesis testing (with all its difficult concepts and which are among the most statistical sins) for problems where there is an interval estimator (confidence, ...
1
vote
1answer
1k views

How to interpret ADF-test optimal lag model reduction?

I was looking at howto figure out the optimal lag order for an ADF-test when I came across this PPT presentation. There are two examples of a ADF-test model reduction, one on p23 and one on p36. I ...
7
votes
1answer
142 views

Can someone help me understand what type of problem I am looking at? Not sure if this classifies as hypothesis-testing

Please pardon me if this question is not clear. I am not sure if I am using the right terminologies. I have conducted an experiment in different environments multiple times. So my data looks ...
5
votes
2answers
10k views

How can we compare multiple proportions from multiple independent populations to evaluate implementation of a treatment?

I want to compare sixteen Case Fatality Rates (deaths per 100 cases) of a particular disease from sixteen different populations across 7 years. Each population received the same treatment but some ...
8
votes
1answer
182 views

Test for independence, when I'm missing one bin of a 2x2 contingency

I'm looking for some help devising a hypothesis test for the following situation. I have a radioactive source that spits out a particle every so often. Also, I have two particle detectors: a red ...
5
votes
2answers
2k views

How to fix the threshold for statistical validity of p-values produced by ANOVAs?

I have run experiments on a group of users under two conditions, measuring the time it took users to finish their experiments. I used a cross-over design where half of the users started in the first ...
20
votes
1answer
3k views

Article about misuse of statistical method in NYTimes

I am referring to this article: http://www.nytimes.com/2011/01/11/science/11esp.html Consider the following experiment. Suppose there was reason to believe that a coin was slightly weighted toward ...
31
votes
1answer
811 views

Are there statistical lessons from the “Bible Code” episode

Although this question is somewhat subjective, I hope it qualifies as a good subjective question according to the faq guidelines. It is based on a question that Olle Häggström asked me a year ago and ...
36
votes
8answers
17k views

Is it possible to prove a null hypothesis?

As the question states - Is it possible to prove the null hypothesis? From my (limited) understanding of hypothesis, the answer is no but I can't come up with a rigorous explanation for it. Does the ...
1
vote
2answers
3k views

Analysing questionnaire data

I want to test if there is a rivalry among two siblings in a family. I have 15 questions in my study and I let my 100 respondents ( distributed equally to two siblings) ranked them 1 to 15. How ...
9
votes
1answer
2k views

What are the practical & interpretation differences between alternatives and logistic regression?

A recent question about alternatives to logistic regression in R yielded a variety of answers including randomForest, gbm, rpart, bayesglm, and generalized additive models. What are the practical and ...
3
votes
1answer
2k views

Statistical significance (chi squared) for percentages over 100%?

So, working in web analytics, it's common to use a chi-square test for A/B tests to test statistical significance. I often use a calculator like this: http://www.usereffect.com/split-test-calculator , ...
2
votes
3answers
601 views

Odds of winning 2 trials in a row

There are 8 people playing poker. So, the odds of winning the entire round = 1/8 2 rounds are played, and Bill wins both rounds. What are the odds this was random? (Hypothesis test?) NullH = Bill ...
4
votes
1answer
819 views

Multiple comparisons on nested subsets of data

Suppose I am doing some experimental procedure on two treatment groups. The procedure has several stages, each of which may fail. Failure at any stage halts the experiment. If all stages are passed ...
19
votes
2answers
57k views

What is the NULL hypothesis for interaction in a two-way ANOVA?

Let's say we have two factors (A and B), each with two levels (A1, A2 and B1, B2) and a response variable (y). The when performing a two way ANOVA of the type: ...
8
votes
1answer
1k views

Two-sample permutation Kolmogorov-Smirnov tests

While it is easier to use the Pearson chi-square/Cressie-Read type test, I would like to test the equality of proportions in $k$ categories across two groups using a Kolmogorov-Smirnov type test of ...
1
vote
1answer
186 views

Removing macro-level time variance

The title might be a bit misleading. Unfortunately statistics is not my area of forte, so gentle guidance along the right path is much appreciated. That said, here's my problem: I'm working on ...
4
votes
3answers
2k views

Statistical significance with sample size of 5 or less

Should small sample sizes of 5 and less be included in reports? This is an area of medical training that is currently being reported only to the course level trainers and statistically not included ...
4
votes
1answer
1k views

Two-sample test for multivariate normal distributions under the assumption that means are the same

Let $\{x_i\}_{i=1}^n$ be a sample from a multivariate Gaussian distribution ${\cal N}(0, \Sigma_X)$ and $\{y_i\}_{i=1}^m$ be a sample from ${\cal N}(0, \Sigma_Y)$. Are there hypothesis tests for $\...
6
votes
1answer
395 views

Nonparametric sign test for correlated variables

I have a question regarding the sign test when the individual measurements may be correlated. Let me start off with some background. Suppose we have 4 Organisms (a,b,c,d),and we make measurements in ...
9
votes
2answers
1k views

How to test if the slopes in the linear model are equal to a fixed value?

Suppose we have a simple linear regression model $Z = aX + bY$ and would like to test the null hypothesis $H_0: a=b=\frac{1}{2}$ against the general alternative. I think one can use the estimate of $...
4
votes
2answers
11k views

Calculating significance of difference between two binary values on one dataset

Suppose my sample comprises 350 instances. Each instance has two independent binary (pos and neg) evaluations, A and B. If my ...
2
votes
2answers
836 views

Aggregation-Level in AB-Tests

Hello fellow number crunchers I hope this a valid question for this forum. I am a lonesome quarter-statistician and have trouble finding someone to ask. Introduction: Wikipedia Explanation of AB-...
7
votes
4answers
6k views

How to test random effects in a multilevel model in R

I have been reading a good book called Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence by Judith Singer and John Willet. The book shows that by modeling in 2 levels, we can ...
6
votes
4answers
978 views

Is there an unpaired version of the sign test?

I wasn't able to find one on Google, and was wondering if one exists...
60
votes
6answers
103k views

What is the difference between a “nested” and a “non-nested” model?

In the literature on hierarchical/multilevel models I have often read about "nested models" and "non-nested models", but what does this mean? Could anyone maybe give me some examples or tell me about ...
11
votes
5answers
15k views

Comparing two cumulative density functions

I am looking for a method to use to test for equality of two cumulative density functions.
12
votes
4answers
4k views

Comparison of the tails of two sample distributions

I have two set of data that are roughly centered around zero but I suspect that they have different tails. I know a few tests to compare the distribution to a normal distribution, but I would like to ...
5
votes
1answer
686 views

How to check that a sample suits multi-dimensional uniform distribution?

I have a 3-dimensional sample $(X_k,Y_k,Z_k), k=1, \ldots, N$ which I suspect to be uniform on some parallelepiped in $R^3$ (i.e. a set of the form [a;b]X[c;d]X[e;f], where numbers a,b,c,d,e,f are ...
1
vote
2answers
2k views

Comparison of two values in a statistically sound manner

For a university paper, I have to compare whether two program versions are statistically significantly different. The comparison in terms of runtime performance is straightforward -- I determine ...
6
votes
2answers
294 views

Estimating the probability of a person getting a question right

We need to estimate the probability of a person getting a question right for a given content area given his history of getting questions in the same content area right in the past. We would also ...
3
votes
1answer
5k views

Uniform Distribution Test

I've got a data-set which I assume is uniformly distributed. Say I've got N=20000 samples and a suspected p=0.25. This means ...
3
votes
1answer
3k views

How to test for parallelism for two linear models?

I'm taking a graduate course in regression analysis and I'm suck on a particular homework question that should be very simple to me! I have the following model: ...
5
votes
2answers
2k views

How to test if change is significant across multiple categories?

Here is a look at my data. We asked the same respondents (n=~400) to provide us with their current and future consumption as a proportion of total expenditure. Plotted here are the mean proportions of ...
2
votes
1answer
581 views

Significance of $r^2$ value

I know about $r^2$ tells you about the amount of variation that can be explained by the predictor variables. I have run a model in which the rsquare has value 0.3010 but has false positive rate of ...
7
votes
1answer
3k views

Effect size of McNemar's Test

I'm using McNemar's test. Basically this question is about best practices when reporting results using McNemar's test. I want to report the effect size. What is a sensible effect size for McNemar's ...
2
votes
2answers
2k views

Resampling, binomial, z- and t-test: help with real data

I am trying to understand how I can use resampling techniques to compliment my pre-planned analyses. This is not homework. I have a 5 sided die. 30 subjects call a number (1-5) and then roll the die. ...
6
votes
4answers
583 views

Estimating the probability that a software change fixed a problem

At work we have a hardware device that is failing for some yet to be determined reason. I have been tasked to see if I can make this device not fail by making changes to its software driver. I have ...
2
votes
3answers
852 views

Using the distance of p-value from alpha

I have two statistical tests which are inverse of each other, meaning that the null hypothesis are reversed. I want to use both the tests to take a decision. For this purpose, I am planning to do the ...