Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

Filter by
Sorted by
Tagged with
5
votes
3answers
1k views

What is the best way to determine if pageviews are trending upward or downward?

Given the following dataset for a single article on my site: Article 1 2/1/2010 100 2/2/2010 80 2/3/2010 60 Article 2 2/1/2010 20000 2/2/2010 25000 2/3/2010 23000 ...
5
votes
2answers
513 views

What to do with p-values when standard errors are obviously biased

What am I supposed to do when I want to interpret significances, although I know that standard errors are biased because of wrong error term assumptions? I know that there is the possibility to use ...
5
votes
2answers
2k views

Is there a way to determine the significance of a change in a d' score?

I have a series of scores in a signal detection task. For each block of scores (i.e. a set of scores from one participant on one day) I have calculated a d' score which I am using as an indicator of ...
3
votes
1answer
301 views

How to compare two groups of patients with a continuous outcome?

I am performing a retrospective study on patients looking at the size of their nostril (continuous variable measured in millimetres) and the need for treatment which is either conservative or surgical ...
1
vote
3answers
2k views

Statistical significance when A/B test has multiple values

I'm sure this is a pretty standard statistics question, but I'm no expert... I'm running an A/B test on my website to see if a change results in users adding more content. So there are 2 basic ...
3
votes
0answers
30 views

Methods for checking if the number of comparisons is sufficient

This question is motivated by a issue regarding network motifs. To determine if a (connected, induced) subgraph $H$ occurs with significantly high frequency in an input network $G$, we generate an ...
8
votes
1answer
417 views

Statistics for gambling machine validation

Problem is that government wants to close electronic roulette and they claim that roulette failed at statistical test. Sorry for my language but this is translated from Slovenian law as good as ...
4
votes
2answers
712 views

What statistical test can I use to detect clumping?

I have time series data that represent dates/times of trades taken in a financial market. I would like to assign a score to this data that represents whether the trades are ...
1
vote
0answers
293 views

Test for biological replicates

I am performing this simple experiment: I have one variety of grass and 8 different fungi (say #1 to #8). I am going to put 10-20 grass plants in each one of 18 containers, and then I will put each ...
10
votes
1answer
1k views

Using kurtosis to assess significance of components from independent component analysis

In PCA eigenvalues determine the order of components. In ICA I am using kurtosis to obtain the ordering. What are some accepted methods to assess the number, (given I have the order) of components ...
4
votes
5answers
242 views

Testing significant effect in 2 by 2 factor design on a binary outcome

I have 4 groups each consisting of 1000 recent college graduates, and I am studying employment immediately after graduation. 2 groups took a business class in college, and 2 groups took a math course ...
7
votes
2answers
2k views

Difference in means in multiple-choice poll

Let's say that N randomly chosen persons where asked a question where the answer could be in either of X categories. For example, 500 persons where asked which of the top 5 political parties they ...
64
votes
3answers
5k views

References containing arguments against null hypothesis significance testing?

In the last few years I've read a number of papers arguing against the use of null hypothesis significance testing in science, but didn't think to keep a persistent list. A colleague recently asked me ...
4
votes
1answer
511 views

Determining which AdWords have the highest amount of user click throughs

Possible Duplicate First, I am not a statistician (though I'd like be one) but I am trying to understand how different tests can be utilized to examine samples. Let's say that I have five of Google ...
5
votes
1answer
785 views

$\chi^2$ test of homogeneity for three-way contingency table

I have a three-way contingency table in which the marginal totals for two sides are fixed and for the third are random. I'm wondering how to perform a chi-square test for homogeneity for such a three-...
5
votes
1answer
2k views

What is the meaning of rank in the context of change-detection?

In a technique that uses CUSUM for change-point detection in this paper, the first step is given below: Let $x_1, x_2,..., x_n$ be the $n$ samples in an event-series. The samples are ranked in ...
6
votes
2answers
2k views

Statistical test for a series of data over time

I have a set of data that looks at the number of "hits" a specific program makes over the course of time. The data goes back to September 2010, and includes data up to March 2011, so the data points ...
59
votes
2answers
22k views

Do we need a global test before post hoc tests?

I often hear that post hoc tests after an ANOVA can only be used if the ANOVA itself was significant. However, post hoc tests adjust $p$-values to keep the global type I error rate at 5%, don't they?...
3
votes
1answer
341 views

Can I split a series of observations of a variable over time into two groups instead of working with time series?

First of all, I’m new to statistics and this is the first time I am trying to apply it to a real world problem. I am doing analysis of a series of observations of a variable over all weeks of a year. ...
3
votes
2answers
2k views

How to test whether correlation measures differ when controlling or not for a third variable?

If the correlation between demographic dissimilarity and satisfaction is $r=.-14$ and the partial correlation, with career development partialled out, between demographic dissimilarity and ...
4
votes
1answer
247 views

How to assess differential risk of disease across three groups after adjusting for other risk factors?

I have 3 different groups: A, B, and C: A (has a medical condition) has 30 entries B (another medical condition) has 31 entries C (control group) has 55 entries All participants have information ...
11
votes
1answer
289 views

Does significance test make sense to compare randomised groups at baseline?

Many randomised controlled trial (RCT) papers report significance tests on baseline parameters just after/before randomisation to show that the groups are indeed similar. This is often part of a "...
11
votes
1answer
2k views

Determining if change in a time series is statistically significant

I have the total number of calls received each week and have plotted them on a chart, going back nearly 3 years. By eye it seems that there was a massive drop over Christmas, that doesn't seem to ...
9
votes
3answers
7k views

What is a good introduction to statistical hypothesis testing for computer scientists?

I was recently exposed to some statistical hypothesis testing methods (e.g. Friedman test) at work, and I would like to increase my knowledge on the topic. Can you suggest a good introduction to ...
1
vote
1answer
1k views

How can I obtain null and residual deviance/degrees of freedom for assessing model significance?

I am trying to determine model significance using: 1-pchisq(null deviance-residual deviance, null df- residual df) I have 5 models: Four models were estimated ...
9
votes
2answers
7k views

Statistical significance of changes over time on a 5-point Likert item

Context: I have two data sets from the same questionnaire run over two years. Each question is measured using a 5-Likert scale. Q1: Coding scheme At the moment, I have coded my responses on a [0, 1]...
12
votes
4answers
7k views

What does it mean for a study to be over-powered?

What does it mean for a study to be over-powered? My impression is that it means that your sample sizes are so large that you have the power to detect minuscule effect sizes. These effect sizes are ...
6
votes
2answers
850 views

How to compare the effectiveness of medical diagnostic techniques?

A pathologist friend came to me for help with the following question for a research project. The goal is to compare the effectiveness of three different diagnostic techniques. The data set is as ...
11
votes
4answers
22k views

Examples of studies using p < 0.001, p < 0.0001 or even lower p-values?

I come from the social sciences, where p < 0.05 is pretty much the norm, with p < 0.1 and p < 0.01 also showing up, but I was wondering: what fields of study, if any, use lower p-values as a ...
4
votes
6answers
653 views

How to assess effect of intervention in one state versus another using annual case fatality rate?

I am a beginner in statistics with just basic knowledge. I have these data: cases, deaths and CFR (Case Fatality Rate-deaths per 100 cases) of a disease for 17 years (1994-2010) from 2 neighbouring ...
3
votes
1answer
2k views

How to do Chow test for constancy of parameters across 2 groups

I am a beginner in statistics and poor in mathematics. I am trying to to assess effect of intervention in one state versus another using annual data. ...
5
votes
2answers
107 views

Test for the effect of UN ratification on corruption in a set of countries

The UN has a convention against corruption (UNCAC) that has been signed by some 140 countries and ratified by most (link). Transparency International publishes an annual report on corruption. They ...
3
votes
2answers
2k views

Choosing sample size to achieve pre-specified margin-of-error

If I want to achieve a margin-of-error of <= 5 % for a representative population sample, how large a sample do I need when: The interviewees are picked from X regions and from Y age groups? That is,...
4
votes
4answers
5k views

2x2 chi-square test vs. binomial proportion statistic

Suppose I'm doing binary classification, and I want to test whether using feature X is significant or not. (For example, I could be building a decision tree, and I want to see whether I should prune ...
2
votes
1answer
2k views

How to determine statistical validity of results

I have two exclusive groups of people and a counter of how many events happened for each group. Lets say group 1 has 7000 people and group 2 has 3000 people. group 1 had 50 events and group 2 had 40 ...
6
votes
1answer
2k views

Interpreting significance of predictor vs significance of predictor coeffs in multinomial logistic regression

I have a multinomial logistic regression with dependent variable valued in {-1,0,1} (reference category is 0) and a number of continuous and discrete predictors. After running the regression a ...
6
votes
2answers
4k views

Which non-parametric test can I use to identify significant interactions of independent variables?

I have an experiment producing results (dependent variables) that don't pass tests of normality, thus I am testing hypotheses using non-parametric tests. My DVs are continuous, while my factors (...
6
votes
2answers
5k views

Poisson distribution and statistical significance

Lets say I have a website which gets 100 hits per day (mu = 100). Yesterday my website got 130 hits (x = 130). If I assume a Poisson distribution, then the probability of getting 130 hits is: ...
5
votes
2answers
554 views

What is the right statistical test to use when attesting claims like “60% people switched over from group A to group B”?

My problem is this: There are 52 people divided into 5 groups: A, B, C, D, and E. From stage 1 to stage 2, they either stay in the same groups, or change groups. The following table shows how many ...
2
votes
0answers
551 views

What is a good statistical test for independent replicates?

I have a series of measurements in which something is sampled and placed into either category 1 or category 2. I have 3x2x3 factors for each set of measurements, and each set of measurements is done ...
8
votes
1answer
151 views

Can someone help me understand what type of problem I am looking at? Not sure if this classifies as hypothesis-testing

Please pardon me if this question is not clear. I am not sure if I am using the right terminologies. I have conducted an experiment in different environments multiple times. So my data looks ...
2
votes
1answer
533 views

Investigate ratios between two groups

I've got some data that has this basic shape (using R): ...
5
votes
2answers
2k views

How to fix the threshold for statistical validity of p-values produced by ANOVAs?

I have run experiments on a group of users under two conditions, measuring the time it took users to finish their experiments. I used a cross-over design where half of the users started in the first ...
5
votes
2answers
770 views

Subsets not significantly different but superset is

I have various clinical data on participants in a study. I'm looking at a continuous variable ("A") and a (binary) categorical variable (group) ("O"). I used a Wilcoxon test in R (the data are not ...
9
votes
2answers
1k views

Assessing significance of correlation

I have two variables, and I can calculate e.g. the Pearson correlation between them, but I would like to know something analogous to what a t-test would give me (i.e. some notion of how significant ...
1
vote
2answers
3k views

Analysing questionnaire data

I want to test if there is a rivalry among two siblings in a family. I have 15 questions in my study and I let my 100 respondents ( distributed equally to two siblings) ranked them 1 to 15. How ...
3
votes
3answers
3k views

Analyzing the difference between two datasets where one is a subset of the other

I apologize in advance for the vague title, but I couldn't think of anything better. I have two datasets, where one is a very small subset of the other. The percentage of people who have a specific ...
1
vote
3answers
337 views

What is a good measure (or set of measures) for the difference between two sample sets?

I'm doing some software testing where we are measuring specific latencies. Generally we run the same test several times to just eyeball the results and make sure that they are consistent across runs. ...
6
votes
1answer
3k views

How to tell if the slope of a line is 0 or there is just no relationship?

I am attempting to examine the change in slope between a predictor and response over two years. In year 1, it is definitely positive. (Linear regression, the 95% CI of the slope doesn't overlap 0). ...
6
votes
1answer
2k views

When is a deviation statistically significant?

In a double-blind study, when are deviations from the control group considered statistically significant? And is this related to the number of samples? I realise that every experiment is different ...