Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

0
votes
0answers
6 views

Significance of difference in means using Monte Carlo Simulation in R

I want to implement a statistical test by the following paper in R: https://journals.ametsoc.org/doi/full/10.1175/JCLI4217.1 Details The above paper calculates the significance of the difference in ...
0
votes
3answers
22 views

Can a prediction be better with insignificant coefficients than with significant coefficients?

I have two OLS models and want to do an out of sample prediction for wages on a test set. In the first model I excluded the insignificant coefficent. The second model has the insignificant coefficient....
1
vote
1answer
101 views

Why do the critical values of the chi-square distribution increase with degrees of freedom?

This makes no sense to me since we usually want more degrees of freedom and the t-and F-distributions reflects this by demanding smaller critical values for significance. The tables I have found only ...
0
votes
0answers
8 views

Why we have to use pooled sample proportion to calculate the pooled standard deviation of the difference?

The context can be found in Hypothesis Test: Difference Between Proportions It says there: ...
0
votes
1answer
22 views

Determine if groups have significantly different metric values

I am familiar with using ANOVA to analyze differences among group means. However, I am interested in analyzing differences for metrics other than the mean. Is anyone aware of a statistical test that ...
0
votes
0answers
26 views

Assistance with statistics question please [on hold]

If anybody is able to assist me with the following question it would be greatly appreciated. Thank you.
0
votes
0answers
21 views

How to perform inference on inverse Mills ratio in Heckit estimation?

I want to estimate log(wages). Most wage estimations suffer from sample selection bias. So I used the two step heckit procedure to correct for it. The problem is that I get an insignificant inverse ...
0
votes
0answers
27 views

Determining whether it matters which subset I pick from my dataset

I'm going to simplify my own research problem with the following example: Lets suppose I have N=40 students in my classroom. I need to select 20 of them to participate in a national math competition. ...
0
votes
0answers
23 views

R: How can I calculate the F-statistic of a logistic model in R?

I am running a logistic regression in R and I noticed that the output does not include the F-statistic which shows the overall significance of the model. In another post, the formula for the F-...
0
votes
0answers
11 views

Probability of rejecting null hypothesis with correlated test statistics

Say I'm working with 2 test statistics and want at least one of them to reject its null hypothesis, so formally: Statistic $T_1$, null hypothesis $H_{10}$ and Statistic $T_2$, null hypothesis $H_{20}$...
0
votes
1answer
15 views

Before/After unpaired study with control villages analysed in R

For the implementation of a project, we have been following household access to services in two groups of villages, one an intervention group (I) and one a control group (C). Access as a Yes/No ...
1
vote
1answer
12 views

if different methods have difference 0.5 in third decimal place while finding accuracy then what to conclude?

For the comparison of different feature selection methods and evaluating performance metrics. After evaluation when their is only 0.5 difference in third decimal place. Is this difference is ...
0
votes
0answers
11 views

Comparing histograms generated by the same process at two different times to see whether they are “significantly different”

I have a dataset that is generated via binning of a continuous variable. The bin sizes are custom and uneven but they are non overlapping and cover the whole relevant part of the number line. A lot ...
1
vote
1answer
20 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
0
votes
0answers
19 views

Significance testing correlations with few data-pairs: how necessary is Fisher Transformation? Alternatives when it appears innapropriate?

Apologies for large post/question: I presented 116 subjects with displays of frequency distributions for 2 groups of data and asked subjects to provide subjective ratings on a 10 point Likert scale ...
0
votes
0answers
25 views

Check significant differences between large samples [closed]

I have two large samples (55540 observations for the first one and 11950 for the second one) where I have calculated three different indices (smallest range: -0.3,0.9; largest: -1,2), each one with ...
0
votes
0answers
16 views

Testing if number of observations in experimental conditions is statisically different

I was conducting in experiment in which i sold products on a pay-what-you-want basis in 4 different information conditions. Apart from comparing the average willingness to pay across the 4 different ...
0
votes
0answers
15 views

Comparing the performance of two algorithms with Wilcoxon Signed-Rank Test

I have two algorithms A and B which have to compute a solution to some problem. Each solution is given some objective value ...
0
votes
0answers
4 views

Should filtering be done before or after correction for multiple testing?

I am trying to find allele-specific bias in some biological data. An equivalent problem that doesn't require understanding biology would be trying to determine which coins, if any, in a giant bucket ...
0
votes
0answers
43 views

What is point of transformation if the effects does not change?

with a question related to transformation. I have done a linear mixed model using the formula ...
0
votes
0answers
16 views

Are the classic assumptions for Chi 2 test still valid?

For a 2*2 table, I want to use Chi 2 test Are the classic assumptions (1# more than 80% of the contingency cells having expected values ­> 5, 2# no zero in the cells ) are still valid? Is existence of ...
0
votes
0answers
12 views

Statistical test for three independent variables and one dependent variable

I use SPSS for my data analysis and cannot seem to think of a proper test to test my hypothesis. I have three independent variables (1. direct emotion expression, 2. indirect emotion expression and 3. ...
0
votes
2answers
24 views

Is this treatment for a very small sample size appropriate?

I am doing a lit review and I am wondering about a study with 12 participants (repeated measures ANOVA with Posh-hoc Bonferroni CI adjustment). There was no control group due to the small n. The ...
0
votes
0answers
17 views

How to choose a significance level [duplicate]

I am a student of statistics and I am learning how to test hypothesis, the main thing I don't understand here is that while choosing a significance level. Almost all of the time when we are testing ...
0
votes
0answers
11 views

4 methods with AUC score mean and std deviation. How can I test which method is the better one? [closed]

I have created 4 algorithms which purpose is to find anomalies. Ive got a AUC score based on the ROC for each method. My question is, what can I say about the results? Ive done 50 experiments which ...
1
vote
1answer
50 views

Statistical test for multiple reponses?

I ran a survey and one question was to indicate which accommodation (hotels, apartment, b&b, ...) respondents prefer when travelling. Thery were allowed to select more than one option. The survey ...
0
votes
0answers
39 views

How to make sure if the residuals are normally distributed? [duplicate]

Hello, I have a fairly big dataset containing 11000 data points. I am doing a ANOVA for my traits. But before that I have to make sure if the residuals are normally distributed or not. So, I did a ...
0
votes
0answers
19 views

Statistical Significance of a Binary Event

Consider a retail bank that makes loans to consumers. Some consumers will default on their loans. Modeling the default event would typically be modeled with a logistic regression if I am not ...
0
votes
0answers
11 views

T-test with conditional data

I have some data giving the vital signs of people before and after one experiment. Also for each single measurement I have some informations about the subject that could affect the results and need to ...
0
votes
1answer
32 views

4 question survery - how to determine statistical significance?

What is the proper approach to determine the highest response rate in a multiple question survey? I did a chi square test of the most popular answer against the 2nd most popular but it feels like that'...
0
votes
0answers
10 views

How to calculate the critical DmodX value from a PCA?

R has a DModX Calculation for PCA. I've seen references in some online texts that a critical DModX can be used to flag samples which are significantly poor fit to the model. The R documentation states ...
1
vote
0answers
7 views

factor of safety

I need to validate a specification on a part. The requirement is to withstand a minimum 1 kg of force (i.e. not fail at less than 1 kg). I was told that if I measure, on 6 specimens, a force of 2.5 ...
0
votes
0answers
15 views

Statistical significance of co-efficient derived through several regressions for panel data

For the panel data that I have, I'm trying to run multiple one-variable regressions (y=ax+e) across time periods - i.e. one for each time period. Naturally, the a's that I find from each regression ...
1
vote
0answers
18 views

Finding first statistically significant point of s curve (sigmoid function) where slope decreases?

Apologies if my title is awful. I have very little statistics experience so I'm hoping the title makes sense. Background on what I'm doing: I'm working on a program that calculates the minimum ...
1
vote
0answers
27 views

How does the frequentist approach to probability estimation work when the number of outcomes is greater than 2?

I've read Checking_whether_a_coin_is_fair and I'm trying to find a resource which generalizes the frequentist approach to the case where there are $k$ distinct outcomes and $n$ trials. Can someone ...
0
votes
0answers
49 views

Statistical test for dependent data- comparing groups of pairs

I have a question about statistical tests for dependent data. This will sound a bit strange/extremely naive but pls bear with me- here is my problem formulation: I have 500 individuals, and I'm ...
0
votes
0answers
12 views

Do p-values indicate a higher level of significance even when $\alpha$ is set arbitrarily?

If you set an arbitrary level of significance $\alpha$ to be some value and subsequently you do an experiment and find that your p-value is significantly less than $\alpha$, is it true that if your p-...
1
vote
0answers
28 views

Bayesian A/B testing and error estimates

Someone from my work started using an AB calculator that they found online and asked my take on the approach which i assume is incorrect. The approach takes the conversion and visits for control and ...
0
votes
1answer
32 views

Understanding p value

Hi i have seen few videos and read some articles as well but i am still confused, i'll trying quoting the example i saw. Helen sells chocolate nutties which claims to have 70gm or more peanuts in a ...
1
vote
0answers
16 views

Dissertation Results. X and Y non-correlation, but then X significantly predictive in regression model. Why? [duplicate]

I just finished my dissertation results and trying to interpret them and very confused. Basically, X and Y are not correlated but then when I put X in my regression model with a few other variables X ...
0
votes
0answers
7 views

which statistical test to conduct when there are two types of responses from same population?

I have negative and positive responses from people who have one of the six types of anxieties. I want to compare negative and positive responses to see if there is a significant statistical ...
0
votes
0answers
8 views

Statistical significance in a sub contingency table

I have a contingency table of data for the twelve months of the year (i.e., a 2 x 12 table), and I want to evaluate the difference between two particular months (the probability of their values being ...
0
votes
0answers
24 views
+50

Significance testing of composite anomalies

I would like to know how to implement t-test for lag composite anomalies. Suppose I have the following 20 points: ...
0
votes
0answers
8 views

Quick test for significant difference in portions (and mean/average) between several group and likelihood?

I have a number of demographic and economic variables – both categorical and continuous – for a few different groups, one is a subset of the other. Estimates derive from IPUMS microdata from which I ...
0
votes
0answers
32 views

How to know which value in a feature is significant

this question might be silly, but how do I know which category (value) within a feature is significant? Say I have some exam result data of students from four different countries: How do I know ...
0
votes
0answers
33 views

Hypothesis testing on a Bernouilli variable

I have a Bernouilli variable which is $1$ with probability $p$. I need to test the hypothesis $H_0:p<\theta$ vs. $H_1:p>\theta$, where $\theta$ is a given constant. The question is to find $n$ ...
0
votes
1answer
23 views

How close to the critical value to be accsepted? [closed]

What if this number 12.32056265 is so close to the 0.05 critical value which is 12.59. Can it still be a significant value with a positive correlation? Or is it then with the critical value of 0.10 ...
0
votes
1answer
22 views

What is necessary sample size for B in A/B test with unequal sample sizes?

I have a situation in which I want to measure if the error rate in system B (test) is worse than the error rate in system A (control). System A has a sample size of 100 and that cannot be modified. So ...
0
votes
1answer
19 views

What model/significance test should I use to see if there's any significance between male and female sexual dimorphism on temperature over time?

In my lab I need to test if there's an significance between male and female sexual dimorphism on body temperature over 40 minutes. The temperature is simulated with a heat lamp and is turned on at 0 ...
0
votes
1answer
9 views

Comparison Tropical Storm Wind Speed

I am investigating wind speed of tropical storms between 2 years. There are an unequal number of storms per year, however want to test if the wind speeds are significantly similar or different. I ...