Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

0
votes
1answer
20 views

How can I determine statistical significance in an A/B test in which the KPI is dependent upon two variables - one bernouli and one continuous?

In my work in online marketing, we frequently run A/B or multivariate web page tests. The Key Performance Indicator for these tests is overall Revenue. The treatment that nets the most revenue, either ...
0
votes
1answer
32 views

Tree structured proportion of success : statistical significance?

I am exploring a tree structured as follows: A/ parent node = parent population, size N, with a proportion of success for an experiment (eg. p = nb success / N) B/ child nodes = sub populations for ...
2
votes
1answer
36 views

Statistical significance in time series and sub-series

So, when I did first year stats in undergrad, we did an experiment where we tampered with a bunch of coins, to see if it would cause a statistical difference in the results. This is a graph of the ...
0
votes
0answers
19 views

Comparing two means and need to know if they are statistically different

I have the net revenue from 81 companies in one year and I'm comparing to the net revenue of the same 81 companies 3 years later. How do I compare (simple subtraction of the means or t-test?), and how ...
0
votes
0answers
11 views

1/F distribution equality significance level

A question about equality $$F_{r_{1},r_{2},\alpha}=\frac{1}{F_{r_{2},r_{1},1-\alpha}}$$ We know if $X\sim\chi^{2}\left( r_{1}\right) ,~Y\sim\chi^{2}\left( r_{2}\right) $ and $X,~Y$ independent ...
1
vote
0answers
7 views

Calculating significance for unequal sample sizes in A/B Testing

I'm running some A/B tests on my site and am using varying visit distributions to minimize risk. Here is an example of one of the test that I have running: Control: 10% of traffic; Variation 1: 45% ...
1
vote
1answer
20 views

When using higher order terms in multivariable regression what is the effect on the P-value

I have a large dataset, lets say 500 lines of data. As a quick workaround to higher order regression I've made educated guesses about what higher order terms I would expect in my regression equation. ...
0
votes
0answers
6 views

significance Compairing two independent groups with single categorical variable

I want to see the significance between two group (One with Albandazole and without Albandazole) inrespect to their CT scane Resolution with three catories(Compltele,partially resolved, Calcified)
2
votes
0answers
17 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
1answer
28 views

Pearson correlation coefficient test: low r and low p

I am running a correlation coefficient test, and the results were: $r = 0.382, p = 2.76 \times 10^{-13}$. So the $r$ value is not that impressive (usually we see $r>.5$), but the $p$-value is ...
0
votes
1answer
18 views

Finding the probability of normality via Anderson-Darling, Shapiro-Wilk , and Kolomogrov-Smirov

I have quite a few distributions that were generated by some system, and I am trying to find the probability of accordance to normality for these distributions. So, the probability of accordance to ...
1
vote
0answers
17 views

Can I estimate Variance of Gamma from Negative Binomial distribution distributed data, given NB is Gamma-Poisson compound

I believe the data I have follows Negative Binomial distribution (over-dispersed Poisson). We know Negative Binomial is a Gamma-Poisson compound distribution. The variance of this Gamma distribution ...
2
votes
1answer
67 views

Hypothesis testing on Poisson Binomial distribution

Imagine $N$ baskets each filled with a different number of products chosen from a set $P$. It is possible to have repeated products in the same basket. For instance: ...
1
vote
0answers
38 views

Can a Linear-Log model be used instead of Robust Standard Errors?

If your regression model has heteroskedastic residuals, one should calculate White Standard Errors that correct for the mentioned heteroskedasticity. If the residuals are also autocorrelated one ...
1
vote
1answer
51 views

Interpretation of Maximum likelihood estimation

I have some problem to interpret the result of MLE estimation : Is it possible to get some advise about how to interpret it? the log likelihood function : $\sum^{n}_{i=1}\log\left( \phi\left( ...
1
vote
0answers
7 views

Fligner Wolfe test and 2-sample Wilcoxon test

What is the significant difference between the Fligner Wolfe test and 2-sample Wilcoxon test? How can I compare the treatment groups with the control groups of different studies after pooling?
0
votes
0answers
10 views

sequential testing and hierarchical tesing

With respect to the hypothesis testing, I once heard sth such as sequential testing, hierarchical testing, and multi-level testing. But I could not find a good reference discussing these topics. Any ...
3
votes
3answers
71 views

Which test for this data set of chemical concentrations

I have a very simple data set: a parameter (concentration of a chemical) was measured at day 0, day 1 and day 2 in three subjects (there is a control group as well, but here all the values are always ...
1
vote
0answers
35 views

Using different letters to show statistical significance

Table 3 of this paper uses the letters A, B and C to show statistically significant differences between variables. I cannot understand what the letters A, B and C actually mean however. Can anyone ...
1
vote
1answer
31 views

Could standardizing an independent variable cause the t-statistic of the OLS estimate to change?

Lets say we are looking to estimate the following standard OLS regression: $y_{i} = \beta_{0} + \beta_{1}*X_{i} + \beta_{2}*Z_{i} + \epsilon_{i}$ and that we choose to standardize $X$ as: ...
0
votes
0answers
17 views

Significant box's test equality covariance matrices, How bad it is?

Having equal number of cases across two groups (60 total cases), if one wants to perform a split-plot Anova and finds out that the Box's Test of Equality of Covariance Metrices is less than 0.001 then ...
2
votes
0answers
38 views

Package ‘fitdistrplus’

I am trying to use the package ‘fitdistrplus’ in R to fit one non standard distribution to my data set. I am trying to copy the methods the package creators used in their tutorial for specifying a ...
0
votes
0answers
20 views

Fisher's Exact Test to assess the significance of a difference between false positive rates

I have trained two binary classification models on the same data and evaluated them using the same test set. For each model I have calculated a false positive rate (and a count of false positives) at ...
1
vote
0answers
30 views

Best statistical test to show significance of two data coming from the same population?

Hope i'm posting in the right thread. I have a sample of genes where I calculate a number of features in the start of each gene, and a number of features in the end of each gene. I plotted the ...
0
votes
0answers
15 views

Mann-Whitney U and KS tests in R are not producing significant results [duplicate]

I wish to compare two test data sets using the Mann-Whitney U and KS tests in R. As a start, I thought to compare two similar sets (an A/A test) expecting to see a highly significant indication of ...
1
vote
0answers
15 views

Intuition For Overlapping Confidence Intervals and Statistical Significance [duplicate]

Two questions: I have a really hard time with the intuition for why the second part of this statement is true: If two statistics have non-overlapping confidence intervals, they are ...
7
votes
2answers
187 views

Can a narrow confidence interval around a non-significant effect provide evidence for the null?

It is obviously fallacious to assume that failure to reject the null implies that the null is true. But in a case where the null is not rejected and the corresponding confidence interval (CI) is ...
5
votes
2answers
148 views
+150

What could be a statistical test for comparing funnel data before and after important changes?

I have some funnel data from a website I manage. After some time I did some improvement on a multi-step purchase process. I wonder if it's possible to formulate a test that with statistical ...
0
votes
0answers
9 views

NPS confidence intervals with bootstrap

I am looking for a method to find confidence intervals for NPS (net promoter score). One method was already suggested here. But I am thinking of using bootstrap. Do you think the following procedure ...
0
votes
0answers
25 views

Non-Stationarity, first differences and Panel Data

I have build a sentiment index and am now validating its statistical properties and significance. I stumbled upon two problems. (1) dfglsindicates that my ...
1
vote
1answer
59 views

Does heteroskedasticity matter if you have a large enough sample?

Let's say you run a regression with over 200 observations. Would this reasonably large sample mitigate the impact of residuals heteroskedasticity as an offshoot of the Central Limit Theorem, or ...
2
votes
0answers
45 views

Test for statistical difference between two curves on same plot?

I've got data plotted for A/Ci curves (plant physiology, A=photosynthesis rate, Ci=internal CO2 concentration--essentially it's a dose-response relationship). Generally, when this type of data is ...
0
votes
0answers
28 views

statistical significance test on cluster analysis result

I've done a hierarchical clustering on a data-set composed by 33 subjects and 2 continuous variables (called V1 and V2), which produces 3 clusters. Now I'm wodering if it make any sense to perform a ...
-2
votes
4answers
71 views

What is relationship between “significant correlation” and “significant difference”

What is relationship between "significant correlation" and "significant difference" between two variables?
1
vote
0answers
11 views

Test which method is better if data is gathered on overlapping time windows (not independent)

I have two methods which are being used to estimate a specific signal. I have a ground truth measurement of this signal and these two methods are using noisy data to estimate this signal. This signal ...
2
votes
1answer
29 views

Standard deviations and significance values

I was wondering if data 2 SD from the mean is deleted as outliers, is it possible there after to report 0.01 significance values? Thanks!
0
votes
0answers
12 views

Classifier accuracy significance assessment in unbalanced train sets

I work with highly unbalanced training sets, and I would like to measure the probability of having certain classifier accuracy by chance in order to measure statistical significances of the accuracies ...
1
vote
0answers
9 views

Long Run VAR alpha and beta significance levels

I am using a VAR with 2 variables and 4 lags. I am combining the coefficients of these variables to get an overall alpha and beta value for in the form $Y = \alpha + \beta X$. In order to get the long ...
0
votes
0answers
27 views

How to model to improve the room usage efficiency based on motion sensor history

To reduce the confusion, I changed my application from traffic to meeting room, so this application is about modeling a meeting room efficiency , the data collection is built by placing a kind of ...
2
votes
1answer
31 views

How do I prove the significance of a non-linear model?

For example, for linear models, you take the p-value of the regression, and you deduce if the regression is or not significative. But with non linear models, in R, there isn't shown a p-value ...
0
votes
0answers
13 views

Determining Significance of Proportions

I've done a test where from a cohort of 112 people I've determined whether they respond or do not respond to the treatment. Now, the definition of them responding or not responding is whether ...
1
vote
1answer
44 views

How can I find a level $\alpha$ most powerful test?

Let $X_1,X_2,\dots ,X_n$ be a random sample from a distribution with pdf $f(x,\theta)$. Find a level $\alpha$ most powerful test of $H:\theta=\theta_0$ against $K:\theta = \theta_1$ when ...
0
votes
0answers
10 views

How do I convert unstandardized RMR estimates into standardized RMR estimates?

I am having difficulty turning unstandardized RMR estimates from AMOS software output in a SEM into standardized estimates to enable suitable interpretation. Can someone give me advice on where to ...
4
votes
1answer
63 views

Statistical significance for comparison of linear regression models

I have two linear regression models (with the same predictors) that try to estimate two different (although related) features of the same population. I am analyzing the hypothesis that these ...
0
votes
0answers
19 views

How Can I Simulate Repeated Significance Testing in R

I work in marketing, frequently running A/B tests on websites to determine which variation is best to serve to site visitors. I would like to be able to run a simulation that highlights the perils of ...
0
votes
1answer
49 views

Hypothesis testing multiple choice question with single answer

I have a survey questions with five options and I ask respondents to pick their single top choice. What test should I use to figure out if the top voted answer is statistically more popular than the ...
1
vote
1answer
49 views

Fisher's exact test or chi-square test

I have a 2x4 table with nominal data (the columns are simply YES/NO, the rows are four categories) ...
5
votes
1answer
39 views

What does the “different populations” result of a significant group t-test mean exactly?

Why does finding a significant difference on a given measure, say mathematical ability, between two groups (e.g., via an independent samples t-test) have to be equivalent to the two groups coming from ...
0
votes
0answers
24 views

Checking if improvement is significant. Test design

I have designed two simple tests to check my algorithm. For a set of 40 pictures, 20 people will judge if: Test 1: Object A or B is seen in the picture (i know the correct answer) Test 2: Object A, B ...