Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

1
vote
0answers
10 views

Individual versus group-wise significance in ARDL context

In an ARDL model approach, what is one supposed to do if the F-bound test shows insignificance while some variables have significant long run and short run coefficients (the error correction term is ...
1
vote
0answers
7 views

test significance for frequencies

I have frequency data for six datasets (3 control samples and 3 treated samples) consisting of values for physical stiffness, when I plot the frequencies against stiffness it looks like the treated ...
0
votes
0answers
13 views

Question about confidence interval

I got curious while I was reading about confidence interval again. Let's say I want to find out about where the person's income stands in distribution by getting confidence interval. Assuming this ...
0
votes
0answers
10 views

How to test statistical significance to prove that a particular technique estimates a variable better than another technique?

Suppose the following is the scenario: I am estimating the measure of a construct. I have Subject Matter Experts (SME) who have manually (through observation) estimated the measures. Suppose I have ...
1
vote
2answers
34 views

Appropriate test for comparison of 50+ groups

I am currently working with a data set containing several hundreds of thousands of instances for which I am trying to find the most appropriate analysis. The goal is to determine whether there are ...
3
votes
1answer
36 views

How do I know if my result is statistically significant?

I wanted to test the effectiveness of 3 different messages I sent to users(respondents) to take a survey. In total, I sent out 6000 messages (3 sets of 2000 users). For message A, I received a ...
0
votes
0answers
3 views

How to test the significance of nucleotide diversities and fixation indices between populations ?

I have four populations with >3Mio Snps and calculated nucleotide diversity as implemented in vcftools in window sizes and then took the mean of it. Now I like to know whether the differences in mean ...
1
vote
0answers
13 views

Controlling for White Matter when looking at a sub-section of White Matter in the Brain - Valid Statistically?

I am interested in other people's opinion on the following matter. I have just come across a brain imaging paper that looks at differences in the volume a sub-section of white matter (SS) in the ...
0
votes
0answers
13 views

Determining minimum required sample size for control (for purposes of measuring lift)

SO in a sales setting where people can either buy or not buy (i.e. binary response), say I have a treatment that I know (or believe strongly) to be effective. I want to give it to as many people as ...
0
votes
0answers
8 views

Relationship between central moments and dependent variable

I was given the task to determine the effects of several statistics on (electricity) derivative prices. Suggestions were variance, skewness, kurtosis to start with. I have a pretty good idea how to ...
0
votes
3answers
67 views

Why do we do hypothesis testing on estimates of linear regression?

I was reading about linear regression and what I understood is that once we minimize OLS equation we get the beta parameters. Its just like solving a normal equation to get the unknowns. Then why do ...
0
votes
0answers
12 views

How do I test the significance of my correlation coefficients for various NONLINEAR regressions?

I have a set of 25 points and I used a TI-84 calculator to calculate the linear, quadratic, logistic, exponential, and power regression models for this data set. I have, likewise, obtained the r^2 ...
0
votes
0answers
19 views

How to test the significant difference of two confusion matrices? [duplicate]

Is there a test for testing the difference between two confusion matrices?
0
votes
1answer
19 views

Test significance of time dummy

I have a logit model $y = \alpha + age + educ + male + year + u$ where y takes the value one if an individual exits unemployment. When calculating the marginal effects of, say, male, after the ...
0
votes
0answers
14 views

How many percent in classification accuracy could be considered as significant

I have 2 classification methods to classify 6 classes. The method 1 achieved the accuracy = A1%, and the method 2 achieved A2%, with A2 > A1. How many percentage point between A2 and A1 could be ...
11
votes
0answers
109 views

Has anyone besides Egon Pearson accessed Gosset's 1904 paper?

Has anyone besides Egon Pearson accessed William Sealy Gosset's 1904 report "The Application of the 'Law of Error' to the Work of the Brewery"? I guess it's Guinness property, but given its historical ...
0
votes
0answers
10 views

Chi squared test on multiple survey responses?

I have completed a questionnaire using a variety of categories (i.e. some questions are yes/no, some are a different categories etc.). I want to find out whether there is a relationship between age ...
1
vote
0answers
13 views

What is way to measure relationship between independent categorical variable on dependent variable (categorical/continuous)? [duplicate]

If both dependent and independent are continuous then we can find correlation and we know which column can be more influential. But if independent variable is categorical, how will i know how much it ...
0
votes
0answers
14 views

significance test for a threshold

I have two samples one of which is significantly larger than the other. I have performed the same operation for both the data sets and I find that the two samples exhibit similar behavior at different ...
1
vote
1answer
48 views

Multiple regression in R results different to simple linear regression? [duplicate]

Hi I am having trouble acquiring the final results for presentation. The results from a multiple regression are different to my results in a simple linear regression. For example, the multiple ...
0
votes
1answer
18 views

What test should I use if:

I'm trying not to have a seizure among the bunch of numbers and data I have. My experiment data consists of three non-equal, non-parametric groups, and I need to compare those groups to state if ...
1
vote
0answers
35 views

Why does the parameter variance change when control variables are added to a regression model?

If I add a control variable to my regression, this changes the variances of the parameter estimates. Why is this the case? Is this because SSE increases and therefore the SSR decreases? So basically ...
0
votes
0answers
5 views

Compare related time-series count data before and after intervention

For a university library we did an intervention to create more free seating spots for students by introducing a way for students to temporarily give up their spot while away. They did this with an ...
1
vote
1answer
19 views

binomial test for two data populations

I have a test that has 39 questions. 20 of them have 18 possible answers and 19 of them have 3 possible answers. I built a QA system that answers 19 of the 20 and 18 of the 19 correctly. I want to ...
0
votes
0answers
15 views

A/B Testing and statistical significance [duplicate]

I often read that when you are running a website conversion a/b test, if you see a 5% significance level before you hit your sample size, you should discount it, and that the more often you compare ...
1
vote
1answer
31 views

Reporting of Heterogeneity in Metaanalysis

The background is a forest plot of a meta-analysis which is reported to have been calculated with a "random effects model". The pooled effect is reported as a standardized mean difference (SMD). ...
8
votes
2answers
110 views

Why is high positive kurtosis problematic for hypothesis tests?

I've heard (sorry cannot provide a link to a text, something I have been told) that a high positive kurtosis of residuals can be problematic for accurate hypothesis tests and confidence intervals (and ...
0
votes
0answers
15 views

What statistical test should I use to compare the binary outcome of two groups (with pre- and post- test)?

I have two groups of people: experiment group and control group. People in each group accepted pre-test and post-test. The outcome of the tests is binary: "good" or "bad". In other words, each people ...
0
votes
0answers
24 views

Which statistical tests are reasonable with this time of day data set?

I'm not really sure which statistical test is appropriate for my data. data: I have behavioral data (being active, resting, eating ...) with a time stamp. Data was taken between 6-18 h and is not ...
0
votes
0answers
72 views

Can I relax the independence assumption for a Chi-Squared test on observational email data?

I'm trying to determine if there is a difference in response rates between email subject lines with the purpose of generating sound hypotheses for A/B testing email campaigns. The data I'm using is ...
1
vote
1answer
38 views

How to determine when the difference between two goodness of fit values is significant?

I have a data series that I'm trying to fit to a model. I'm trying several types of models (exponential, linear, logarithmic). In order to assess which one it fits best, I use a Residual Sum of ...
0
votes
0answers
13 views

Assessing significance / confidence of a crossvalidated performance measure

I have a prediction model $P$ and I use some performance measure $I$ to measure $P$'s accuracy. The distribution of $I$ is unknown (it's a custom metric, which is somehow similar to the precision ...
0
votes
0answers
13 views

Chi squared null-hypothesis test. Long tail

I have the following data. ...
0
votes
0answers
7 views

Significant change over time between categories

I am trying to establish whether change over time is significant at an archaeological site. Basically, there are seven different time periods (lets call it A-G) and in each time period there are six ...
3
votes
1answer
30 views

Clustering and A/B testing

My question is the following: Let's imagine I've defined clusters in my data (different segments of customers) and I run an A/B test. Can I compare the performances of the different clusters on the ...
1
vote
1answer
28 views

Check population randomness in an A/B test?

I am fairly new to A/B testing and I don't really understand some points that my colleagues make. On our website users are randomly assigned to A or B when their session start (by randomly assigned, ...
0
votes
0answers
16 views

Test to test for significant differences in attrition rates

Suppose I have two populations and I want to compare the attrition rates between them to see if there is a significant difference. Attrition rates are tallied up monthly. For each individual in the ...
0
votes
0answers
38 views

If my goal is to show very low correlation, how should I check for statistical significance?

I am working on a research project in social sciences. My null hypothesis is that there is a high correlation between two variables and my alternative hypothesis is that there is very low to no ...
0
votes
0answers
12 views

how to know if the observed overlaps in independent sets is significant?

I have a list of names ( derived from a statistical test ) that overlaps with a list of names ( set2 ). I generated a random list of names ( random ) and overlapped with set2. How do I know if the ...
4
votes
1answer
79 views

Can someone explain this cartoon? [duplicate]

I ran 160 regressions on various combinations of data sets, predictors, and dependent variables. I am now trying to sift through the results and separate good models which are "real" versus those ...
0
votes
0answers
15 views

Significant test using Rapidminer tool

I want to execute Statistical significance test on a dataset. I have 5 classifiers (K-NN, SVM etc) . I want to use Anova and T-Test using Rapidminer tool. Please let me know following: Is this ...
1
vote
0answers
29 views

p-value in interpreting path coefficients in path modeling of PLS-SEM

I'm beginner at statistics, so my question may seem to have non sense for some. I was trying to interpret a PLS-SEM results, which is a kind of path modeling. The results of path coefficients are ...
0
votes
0answers
20 views

Smaller p-values on a selective multiple regression analysis [duplicate]

When I run a multiple regression analysis in Excel on 20 independent variables and 1 dependent variable (in two goes), I obtain in the summary a set of p-values. When I select the (six) ones that are ...
1
vote
0answers
43 views

How can I determine whether a change is statistically significant ONLY based on confidence intervals and points of estimate?

The table above shows the proportions (%) of a given country's population that have experienced theft from vehicles each year along with respective confidence limits. How can I tell if a change ...
0
votes
0answers
11 views

Alternative to Pearson's Chi Squared for Speech Synthesis Preference Listening tests

I have a simple Preference Listing test: 24 Participants, 10 preference questions each. each participant answers 10 times (once for each of my 10 stimuli pairs). This means 240 data points overall. ...
0
votes
0answers
10 views

Correcting for categorical differences in a correlation between descriptive groups and another variable

What I want to do it work out the effect of method of assessment on different students marks. So for example Module 1 has 70% exams. Students with qual type A got marks 1,2,3,6 while students with ...
1
vote
0answers
26 views

How to determine if a (binary) variable has a statistically significant effect on a response variable?

To be more specific, assume the following scenario: A customer advertising for a job on your site wants to know whether buying an extra ad-product or not will increase the number of applicants. You ...
0
votes
0answers
9 views

Normality test - classification accuracy from multiple classifiers

I would like to employ statistical tests to test if the difference in classification accuracy is significant. I first start with the normality test of the data set. The problem is I am not sure ...
0
votes
0answers
27 views

How to interpret P-value from mixed model and P-value from overall ANOVA?

I fitted mixed effect model with several effects with lme package in R by command ...
0
votes
0answers
23 views

Statistical significance test for comparing machine learning algorithms

I have to find statistical significance test of machine learning algorithms SVM, Random Forest and K-NN. I am using data sets (having 10 classes). I am using rapidminer for experimentation purpose ...