Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

0
votes
1answer
5 views

Which test of statistical significance for 2 variables (categorical data), but with more than 2 categories on one of the variables?

So I'm working on a project, one aspect of which looks at the effect of nationality (my independent variable) on the tendency to quit a job (my dependent variable) and I can't figure out which test to ...
0
votes
0answers
5 views

Comparing the slopes from two models

I'm working on investigating the results of two models of a dataset. The dataset has several thousand data points (representing daily measurements over the course of several years). The data have a ...
0
votes
0answers
15 views

Can I Use Distance Matrix (Weighted) For GLM Fit (Using R)

I am new to statistics but a business user liking to base my decisions on prediction models. And I am relatively new to regression. I have data in the below format. (I have used only imaginary ...
0
votes
0answers
4 views

what is the relation between significance level and forecasting values

From the past data we estimated the future values in R with default level of significance but my doubt is if we increase or decrease the significance value then how much change will be there in ...
1
vote
1answer
23 views

Forecasting Skew & Kurtosis

I have seen several models for estimating the expected value and variance of a distribution. I am curious, to learn if anyone has looked at models that extend beyond these first two moments ...
0
votes
0answers
19 views

Size of parliament in Lottocracy? [on hold]

Assume you are living in a country with a population of 100 million people. In this country the members of the parliament are not selected by vote but by lot (see Lottocracy). How many members would ...
2
votes
0answers
14 views

Two experiments with shared control, can I generalize?

I have run two experiments, let's say the control condition is A. The two treatment conditions are B and C. I determined that B < A on some variable, and that C > A on that same variable. These ...
0
votes
0answers
7 views

Finding significant values in my tree-like distributed data

i have a question about my data. Let me first describe what i know about it. I know that the number of data points I have is large (200-1000) I know each data point value is greater or equal to 0 I ...
1
vote
3answers
163 views

Small data set and linear regression

If I have a small data set (30 samples) am I more likely to obtain a statistically significant result? From my understanding if there are any relationships within the data set then they will be over ...
1
vote
1answer
37 views

Finding significant higher values among the entire population sampling

We are developing a software product that monitors user interactions with some website. Each interaction is referred as a hit. One of the things we measure is how long it takes for the user to get the ...
0
votes
0answers
8 views

Using LIWC for ANOVA and pairwise analyses

I want to use a tool called Linguistic Inquiry and Word Count (LIWC) to analyze text data. As far as I know, it counts the number of words found in a given text that match a set of dimensions (e.g., ...
0
votes
0answers
16 views

Perceptual Agreement Statistics - Test for similarity between pairs?

I'm struggling to identify if there is a suitable and established test for determining if pairs formed from two separate populations are statistically similar when multiple people attempt to match ...
0
votes
0answers
4 views

Calculating an SLA and considering statistical confidence

When calculating a Service Level Agreement (SLA) one might consider the %age of successful transactions with the system. E.g. if 99 out of 100 transactions fail, the 99.9% SLA was not met. However, ...
0
votes
0answers
10 views

1 sample z test of proportions

We are trying to work out whether what we currently do is accurate. We want to work out if a proportion (% scoring 5 on a 5 point scale) are significantly below a target score. Normally a 1-sample ...
1
vote
1answer
39 views

Significance of continous variables

I have a table with several numerical continous measurements and an indicator column catastrophy ...
22
votes
3answers
2k views

Is p-value essentially useless and dangerous to use?

First are some background information. This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that [Bayesian statistics] is proving ...
0
votes
0answers
28 views

Statistical Significance of a Sample [closed]

Background: I do not know anything about the population except the population size (N). All I have is a sample. Question: How to I test if this sample is statistically significant (e.g. 95% ...
0
votes
0answers
16 views

Joint distribution R [migrated]

I have a list of 10 stocks, with each having a time series of log returns (AIG, JPM,...). I have calculated the log returns for each of the stocks as follows: ...
0
votes
0answers
10 views

Adding a new factor to an existing factor model for stock returns

Let's assume I have a three factor model of stock returns (Cross-sectional model): $R_i = \alpha + \beta_{1,i} X_1 + \beta_{2,i} X_2 + \beta_{3,i} X_3 + \epsilon_i$ for $i =1,..,N.$ where ...
0
votes
0answers
5 views

Sensible metric for describing an item's position in a power law distribution?

I am responsible for monitoring quality on a large website with many tens of thousands of pages. We have a metric for tracking the quality of content that is "number of problem reports raised per ...
2
votes
1answer
45 views

One model performs better than the other. How to measure if it is statistically significant?

So, let's say that I train two models on the same dataset. I run the experiment once and I get the following results: Using a Neural Network I get an AUC ROC of 0.941. Using Random Forest I get an ...
1
vote
0answers
15 views

Multiple ANOVAs, then multiple Tukey HSDs, correcting for multiple comparisons

My experiment has 5 conditions (let's say Treatment A, B, C, D and E). There are 12 measurements (let's say Measurement 1-12). I'm running 12 ANOVAs on the measurements, I assume this means I should ...
2
votes
2answers
31 views

When to create a control group with paired T test

So if I wanted to measure the effect of a drug on a group of people, I think I would use a paired T test based on my research. But what if something caused a change in my group that I did not account ...
2
votes
2answers
276 views

Is “Confidence Level” just 1 minus P-Value?

If the results of a Chi-Square test give a P-Value of 0.01 then can we say that the confidence level in their being a difference is (1-0.01) = 99% confidence. Background. We are conducting website ...
4
votes
3answers
110 views

Why do irrelevant regressors become statistically significant in large samples?

I am trying to better understand statistical significance, effect sizes and the like. I have a perception (perhaps its wrong) that even irrelevant regressors often become statistically significant in ...
0
votes
1answer
41 views

Selecting variables in multiple linear regression in R

Consider that we have a problem with 4 variables (y, x1, x2 and x3) and we want to do a multiple linear regression model. As we need to know which variables are the most important in the problem, we ...
0
votes
0answers
16 views

Significance Test for two groups, and two time periods (Chi Squared?)

Apologies but as I am not well-versed in statistics, this might work best if I describe my data.Thanks for your patience. The exercise is that I'm testing to see if an advertising campaign had a ...
1
vote
1answer
15 views

Significance of differences in answers to vague questions

Participants in an experiment were divided in “Truth-judgers” and “Falsity-judgers”. The former were asked “When is it true to say that a man is tall [cm]?” while the latter were asked: “When is it ...
0
votes
2answers
22 views

Comparing within-subject z-scores (survey data)

I have some messy survey data, wherein one group of interest (cut on one self-reported behavior) rated every single attribute (7-point Likert scales) higher than any other group. I think this has ...
0
votes
0answers
18 views

applying statistics in “filling in” a data set?

Let me first say that my professional role is that of a mid level OLAP/OLTP developer (lesser experence with OLTP). Ive only had business statistics in school many years ago and I am currently taking ...
1
vote
2answers
71 views

Statistical significance versus sample size [duplicate]

Statistical significance of a (variable in a) model grows with sample size. Citing Gilbert (1986): If one uses test statistics with constant size (i.e. a constant degree of confidence), almost any ...
1
vote
0answers
27 views

Significance of differences between answers of truth-judgers and falsity-judgers

Participants in an experiment were divided in “Truth-judgers” and “Falsity-judgers”. The former were asked “When is it true to say that a man is tall [cm]?” while the latter were asked: “When is it ...
1
vote
1answer
14 views

How do you develop Confidence Intervals based on non average data

I am collecting votes from an election poll. The rules for voting allows the voter to select ten out of 36 candidates for election. Any candidate whom receives at least 75% of the vote or more wins ...
4
votes
2answers
85 views

Testing if frequency of event occurrences over week days is uniform

I’m trying to figure out if there’s a calculation that can be done to show you have enough data to draw conclusions on trends within the data set. I have made an observation on a class of equipment ...
3
votes
0answers
31 views

Which statistical test should I use to compare the median values between two groups in my study?

I am doing a retrospective study comparing the effect of medication adherence on the length of a patient's illness. One group of patients took their medication compliantly, and I want to compare the ...
1
vote
1answer
37 views

paired t-tests and interpretation of results

My study is about comparing the scores a group of students (sample, N=14) in two different conditions. They play the same game in a tablet and using a computer. They were administered with a ...
5
votes
1answer
65 views

when can I say there is a relationship between events?

I am analysing a weather data for a computer science course using hadoop.I am taking away all the weather related issues out of the analysing and just looking strictly at it in the eyes of statistics. ...
0
votes
0answers
35 views

How to tell if the occurence of a variable is statistically significant/meaningful?

Since my previous question seems to be unclear, I will try to put it in other words. I have been working on a child speech corpus, which consists of sentences in which children of various ages ...
4
votes
2answers
125 views

Alternative to T-Test

I have a series of values and I need to test if the returns are > 0. The t-test assumes normality, so I was wondering if there are other alternatives. Note that the standard suggested sample size is ...
1
vote
0answers
4 views

Significance of proportions in clustered data

I have data for multiple homes, each of which spends a certain amount of time in Condition 1 and then a certain amount of time in Condition 2. While in these two conditions, there are a variable ...
3
votes
0answers
17 views

Measuring “almost the same” when significantly different

The following table is of UK domiciled university applicants to UK universities in $2013$ and of acceptances, grouped by self-declared disability. A $\chi^2$ test on the numbers shows an extremely ...
0
votes
0answers
16 views

how to measure the similarity or overlapping significance between two character vectors with pre-ranked items?

Here there are two character vectors (like V1 and V2). Items in each vector are unique, which means no duplicate in each vector. All the items in the vectors come from a known set (#item in set >> ...
1
vote
0answers
15 views

How to calculate confidence intervals for Precision & Recall (from a signal detection matrix)?

I built a detector to detect a binary outcome and then took a random sample from the population. From this, I can create a signal detection/confusion matrix (hit, miss, false alarm, correct ...
1
vote
0answers
37 views

BINARY or ANOVA test in R

I'm confused on what test to use for an experiment I have conducted over the past months. Let me give you some context: I ran a germination experiment, sowed 900 seeds total in trays of 40 in a ...
2
votes
1answer
31 views

Significance testing questionnaire answers

I apologize, there may be an answer for this but I couldn't find it. I have a survey of 1000 people, and for a particular question they were asked which do you prefer of A, B & C. 44% answered A, ...
5
votes
3answers
783 views

How to choose a confidence level?

I often use a 90% confidence level, accepting that this has a greater degree of uncertainty than 95% or 99%. But are there any guidelines on how to choose the right confidence level? Or guidelines ...
1
vote
1answer
45 views

Hypothesis testing with group of stocks

I'm in a situation where I need to compute a p-value relating to the hypothesis that the mean daily return from a strategy applied to a group of stocks is different from zero. The usual way to do this ...
1
vote
0answers
19 views

statistical signficance and sample size

I have a question regarding statistical significance and sample size for a website a/b test. using a sample calculator and knowing my control's conversion rate, 80% statistical power and a minimum ...
1
vote
1answer
28 views

Statistically significant comparison of classifiers

I am working on a movie review sentiment analysis project, and comparing various classifiers on the same dataset. The data for the two classes is balanced, so I'm using accuracy on 3-fold cross ...
0
votes
0answers
14 views

Comparing a sample to a known population

I am trying to compare a single sample to a known population to see if that value is significantly outside of the population mean. As an example, I know my height and I know the height of 10 of my ...