Questions tagged [statistical-significance]
Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).
6,385
questions
1
vote
0
answers
16
views
Statistical tests for two rates
For a paper I am writing, I am comparing influenza burden in two locations. For each location, I have the rate of symptomatic influenza infections per 100,000 population, as well as a 95% confidence ...
1
vote
1
answer
26
views
Poisson regressions in ratio: why is the counterpart not significant?
I am not a statistician and have limited knowledge about the underlying mathematics behind models but I am curious about something I found. I have count data, something like this: out of 150 words in ...
0
votes
1
answer
24
views
Test for significant difference between two groups measured across time
I have two groups of time-dependent samples whose measured value changes nonlinearly with time. There are 3 samples in group A and 3 samples in group B. An example of my data is illustrated below.
...
0
votes
0
answers
31
views
A/B testing impact of two different product pricing strategies on revenue
Scenario
I am running a marketplace with 100,000 products available for purchase on my website. In most months between 1000-10000 different products are sold. This means that in any given month the ...
4
votes
1
answer
409
views
Model assumptions - not worth the effort?
This question was inspired by this discussion I read recently. After obtaining our results, the assumptions from the models we used should be checked, otherwise these results may be deceiving.
...
0
votes
0
answers
27
views
Comparing averages of two groups
Say I want to check if the averages of 2 groups (A and B) are the same.
For each member in group A, there are some observations; same applies for group B.
If I want to take the average of A and B, I ...
0
votes
2
answers
18
views
Define metric to use after ab test is finished
Let's consider such a setup for an experiment: I've made some new feature to the existing system and want to test, whether this feature will bring positive value to the system. May be I don't know ...
1
vote
0
answers
50
views
Statistical significance of hierarchical (multi-level) associations
Association rule mining is a common technique in Data mining used to find itemsets which appear together very frequently in large transaction databases (e.g. in Table 3 we see that in a transactions ...
1
vote
1
answer
10
views
Estimating Marketing Contribution from Opted-In Users
I'm trying to estimate how much marketing contributes to the usage of our product, but because of consent requirements, I can only collect data on a subset of users - the ones who have consented to us ...
0
votes
1
answer
21
views
Relationship between p-value and Cramer's V
Is there a relationship between p-value and Cramer's V value after conducting Chi-square test for independence... is for Cramer's V value of about 0.13 the associated p-value is of about 0.001 or it ...
8
votes
4
answers
420
views
Is p-value of R-squared and adjusted R-squared a thing?
I'm currently reading a paper which utilises a multiple regression and reports the adjusted $R^2$ with a p-value, and I'm wondering what this p-value refers to. Can you calculate p-values for adjusted ...
1
vote
0
answers
28
views
emmeans differences in logistic regression give wrong significance in case of complete separation
Suppose (fictitious example) we have a logistic model for dependent "Happy" (0/1), and factor "Age" ("young", "middle", "old") as independent variable ...
1
vote
0
answers
19
views
Can p-values from permutation tests be interpreted as a measure of effect size between group?
I'm conducting a research project involving gene expression analysis.
We've developed a method to calculate a score from a given set of genes, A.
To evaluate the ...
1
vote
1
answer
90
views
multiple testing adjustment for fisher exact test
I have 4 groups (G1, G2, G3, G4) consisting of samples of biological material, and we test them in order to identify are there any bacteria present. We also know that all of the samples contain virus ...
0
votes
1
answer
38
views
How to compare a single (counted) value to a mean for statistical significance in R
I am working on a project estimating salmon escapement through visual spawner surveys. We conducted weekly surveys and counted the number of individuals we observed. These numbers were used in an Area-...
0
votes
1
answer
50
views
What does it mean for a variable to be statistically significant but not selected as important by classification methods?
I'm basically playing around with some lipidomics data to practice, so my question is purely theoretical.
I wanted to see if I could find lipid classes that differ between two groups and I was ...
1
vote
0
answers
28
views
Is it valid to do a Bayesian analysis on an old paper which didn't publish a Bayesian analysis? [closed]
Is Bayesian Analysis, and the judgement it allows on the chance of a true effect being significant despite an overall analysis being not statistically significant, something that can be ...
0
votes
0
answers
8
views
Better Prediction model with large DIC value
I am using Integrated Nested Laplace Approximation (INLA) to predict birds population.In my (INLA) models, I've noticed that when I include auto spatial correlations, my predictions are good and make ...
0
votes
2
answers
34
views
How many z-scores should be used when determining a p-value?
Suppose one wants to test whether the average sum of tickets in a box is X.
Since the z-score is computed by sampling N tickets and then seeing how many SE's away the average is from the expected ...
0
votes
0
answers
32
views
Which test do I need for testing differences in two populations?
I would like to compare the different variables in a life table of two populations and check if the differences are significant. However, I am totally unsure which test is appropriate.
For example, I ...
0
votes
0
answers
13
views
How to compare two distributions of Coefficients of Variation?
For my master's thesis (machine learning) I am studying the effect of using a certain type of classifier on the stability of feature importance techniques (SHAP & LIME) under different levels of ...
1
vote
0
answers
11
views
How to Evaluate Interaction Effects in Propensity Score-Matched Samples:
Suppose I want to study the association between an exposure X and outcome Y, and I have used the propensity score to match each exposed subject with those unexposed but with similar characteristics ...
1
vote
2
answers
69
views
Significant interaction when only one of the two main effects is significant? [duplicate]
I have run a mixed effect logistic regression model, and I have obtained a significant main effect of Factor A, while no significant main effect of Factor B. However, the interaction between Factor A ...
1
vote
1
answer
22
views
What statistical analysis method should I use for two independent variables with interval data and a dependent variable with continuous data?
I'm doing a study on the potential of date leaf ash and date fruit seeds as replacement materials for cement and sand, respectively. For date leaf ash (DLA), there is 0% replacement and 5% replacement,...
0
votes
0
answers
40
views
How to statistically test an increase?
I have two temporal networks in which I report their number of nodes and the average degree in different time intervals:
time
Nodes_Network_1
Nodes_Network_2
Average_degree_Network_1
...
0
votes
0
answers
18
views
The difference between significant differences pased on positive and negative ranks on Wilcoxon signed ranked test [closed]
I am wondering about the interpretation for the results of Wilcoxon signed rank test. Now I am comparing between three predictions made by three different sets of feature and I want to test the ...
2
votes
1
answer
39
views
Can a result be statistically significant if the null value is one of the 95% confidence intervals?
I'm reading a paper on the effect of 2 different drugs on mortality. The results are reported as RR 0.96 (0.94-1.00), p=0.03. The p value indicates significance but the upper 95% limit is 1.00.
Am I ...
4
votes
2
answers
156
views
Significant variable and very low R-squared [duplicate]
I'm testing if economic growth before an election is correlated with vote percentage in elections. So I have one independent variable in my model.
The problem is: my independent variable is ...
1
vote
1
answer
21
views
How many samples should I test to be 95% sure that no error exists?
If I have a million population of products and I will tolerate no error in them. How many samples should I test to be 95% sure that no error exists?
I am new to statistics. I know you might need some ...
0
votes
0
answers
9
views
Which variables should I include/exclude in my regression analysis for GDP? [duplicate]
I am wanting to establish the impact mobile money has on Kenyan economic growth. I am building my regression model and have collected data for GDP, consumption, government spending, investment, net ...
0
votes
0
answers
42
views
Statistical Tool to Compare Raw Data to Lower Resolution Data to Determine the Resolution When Data Is Lost
I have dimensional data collected from a product every 1 second. The product runs can often last days and analyzing the entire dataset can be difficult due to the large amount of datapoints. The ...
2
votes
0
answers
26
views
T-tests for differences of slopes of linear regression when intercept is forced through 0
I am looking to compare the differences in linear regression slopes between two datasets and at first I used a two sample t-test. However, when I force the intercept through $0$ for both regressions (...
3
votes
2
answers
58
views
Effect of interaction term: what if one of the main effect is not statistically significant?
Suppose we have the following Poisson regression model:
$\log(y) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2$
Where, for example,
$y$ is the number of children in a family,
$x_1$ is the ...
0
votes
0
answers
23
views
Statistical significance between sigmoidal regression curves
I have a data-set of weight ~ temperature that was measured in 3 species (A-B-C), each consisting of 2 genotypes (a-b) that were grown in 3 soils (1-2-3) each. Weight ~ temperature for each Soil x ...
4
votes
1
answer
31
views
When testing significance between a regression coefficient of two data sets using a Z-test, how to interpret the z-value?
I am following the methods described in:
Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, ...
5
votes
1
answer
146
views
Statistical tests for data on occurrence of reptiles found under different types of artifical cover objects
I am studying small-bodied ground snakes and skinks (lizards). I use artifical cover objects to determine their presence.
Artificial cover objects (hereafter CO) consisting of 7 different types of ...
0
votes
0
answers
35
views
Number of levels for ordinal DV, variance, and statistical significance
I am interested in how closely a person places themselves to a certain situation in the responses they give. The responses are coded, and I create a score, with the lowest value indicating the person ...
3
votes
2
answers
85
views
How do I work out the significance of main effects in negative binomial models with more than two factors?
I am trying to look at how the number of events X is affected by the three factors A (4 levels), B (2 levels) and C (2 levels) using a negative binomial model as follows:
...
2
votes
1
answer
62
views
Statistical test for binomial draws with unknown number of trials
I have two binomial random variables $X_1 = \text{Bin}(N, p_1)$ and $X_2 = \text{Bin}(N, p_2)$; same number of trials but different probabilities of success. Suppose that the number of trials $N$ is ...
5
votes
2
answers
575
views
How does reducing my alpha-threshold to 0.005 affect my required sample size?
Among statisticians there is an ongoing debate whether the long-standing "standard" threshold for declaring the outcome of a statistical test as "significant" (a.k.a. alpha-level) ...
0
votes
2
answers
88
views
Null hypothesis in clinical trial question
Consider a clinical trial where we want to study the evolution of the occurence of some event during two time periods, with and without treatment.
We know that, not only a difference close to 0 won't ...
0
votes
0
answers
10
views
How to compare peak location and tail length of two different distributions?
I have the distributions of the fraction of people in each income bracket in a town in 1990 and 2020. The total sample size is the same in both, and assume that the incomes have been adjusted to ...
4
votes
2
answers
166
views
Which statistic to use for measurement between two samples [closed]
I am measuring stem widths of a control plant and a mutant plant and want to determine if there is any significant difference between them. The sample sizes are not the same. When I use a test like ...
3
votes
1
answer
24
views
Appropriate method for seeing how similar two items on a survey are?
I have a survey and we want to look at a pair of questions we think are very similar and possibly asking the same thing. If they are we'd like to reduce the two questions to one question. Both ...
5
votes
3
answers
432
views
Hypothesis testing for detecting a (damped) sinusoidal signal in noise
I have a signal in white noise that has the following form:
\begin{equation}
r[t_i] = A e^{-t_i/\tau} \sin{(\omega t_i + \phi)} + n[t_i]
\end{equation}
I would like to test whether the signal (1st ...
0
votes
0
answers
28
views
Which test would I use in this situation?
I have a sample of people, $n=300$. All $300$ should be offered both of two therapeutic treatments (Treatment A and B). I will be collecting data on how many people were offered A only, B only, and A ...
3
votes
1
answer
74
views
Remove non-significant independent variabels and re-run multiple regression
We have done a multiple regression with the 3 Theory of planned behaviour variables (subjective norm, perceived behavioural control and attitude) plus 2 added variables (because previous research ...
0
votes
0
answers
20
views
Structural break test, individual regressors
Say I have an equation
$y=\alpha_0+\alpha_1 x_1 + \alpha_2 x_2$
and I'm testing for a structural break (SB) in one of the slope coefficients. How would I discriminate between the breaks. For example ...
5
votes
4
answers
941
views
Statistically significant difference
I am reading a medical paper, where it says:
There was a statistically significant improvement in dyspnea in the IPC group at 6 months, with a mean difference in VAS score between the IPC group and ...
0
votes
1
answer
47
views
Statistical testing on non-random sample?
I am working with a non-random sample in an observational study but would like to do statistical testing to show certain trends and processes. Statistical tests assume random sampling, which is not ...