Questions tagged [statistical-significance]

Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).

Filter by
Sorted by
Tagged with
1 vote
0 answers
16 views

Statistical tests for two rates

For a paper I am writing, I am comparing influenza burden in two locations. For each location, I have the rate of symptomatic influenza infections per 100,000 population, as well as a 95% confidence ...
Elizabeth's user avatar
1 vote
1 answer
26 views

Poisson regressions in ratio: why is the counterpart not significant?

I am not a statistician and have limited knowledge about the underlying mathematics behind models but I am curious about something I found. I have count data, something like this: out of 150 words in ...
Giuseppe Magistro's user avatar
0 votes
1 answer
24 views

Test for significant difference between two groups measured across time

I have two groups of time-dependent samples whose measured value changes nonlinearly with time. There are 3 samples in group A and 3 samples in group B. An example of my data is illustrated below. ...
bjarke15's user avatar
0 votes
0 answers
31 views

A/B testing impact of two different product pricing strategies on revenue

Scenario I am running a marketplace with 100,000 products available for purchase on my website. In most months between 1000-10000 different products are sold. This means that in any given month the ...
qoga's user avatar
  • 1
4 votes
1 answer
409 views

Model assumptions - not worth the effort?

This question was inspired by this discussion I read recently. After obtaining our results, the assumptions from the models we used should be checked, otherwise these results may be deceiving. ...
JED HK's user avatar
  • 409
0 votes
0 answers
27 views

Comparing averages of two groups

Say I want to check if the averages of 2 groups (A and B) are the same. For each member in group A, there are some observations; same applies for group B. If I want to take the average of A and B, I ...
Andrei's user avatar
  • 1
0 votes
2 answers
18 views

Define metric to use after ab test is finished

Let's consider such a setup for an experiment: I've made some new feature to the existing system and want to test, whether this feature will bring positive value to the system. May be I don't know ...
Joitandr's user avatar
  • 111
1 vote
0 answers
50 views

Statistical significance of hierarchical (multi-level) associations

Association rule mining is a common technique in Data mining used to find itemsets which appear together very frequently in large transaction databases (e.g. in Table 3 we see that in a transactions ...
gis_grad_student's user avatar
1 vote
1 answer
10 views

Estimating Marketing Contribution from Opted-In Users

I'm trying to estimate how much marketing contributes to the usage of our product, but because of consent requirements, I can only collect data on a subset of users - the ones who have consented to us ...
Ben's user avatar
  • 11
0 votes
1 answer
21 views

Relationship between p-value and Cramer's V

Is there a relationship between p-value and Cramer's V value after conducting Chi-square test for independence... is for Cramer's V value of about 0.13 the associated p-value is of about 0.001 or it ...
Rusul Raed's user avatar
8 votes
4 answers
420 views

Is p-value of R-squared and adjusted R-squared a thing?

I'm currently reading a paper which utilises a multiple regression and reports the adjusted $R^2$ with a p-value, and I'm wondering what this p-value refers to. Can you calculate p-values for adjusted ...
mrepic1123's user avatar
1 vote
0 answers
28 views

emmeans differences in logistic regression give wrong significance in case of complete separation

Suppose (fictitious example) we have a logistic model for dependent "Happy" (0/1), and factor "Age" ("young", "middle", "old") as independent variable ...
BenP's user avatar
  • 1,019
1 vote
0 answers
19 views

Can p-values from permutation tests be interpreted as a measure of effect size between group?

I'm conducting a research project involving gene expression analysis. We've developed a method to calculate a score from a given set of genes, A. To evaluate the ...
zhang's user avatar
  • 113
1 vote
1 answer
90 views

multiple testing adjustment for fisher exact test

I have 4 groups (G1, G2, G3, G4) consisting of samples of biological material, and we test them in order to identify are there any bacteria present. We also know that all of the samples contain virus ...
ariadnaoliver's user avatar
0 votes
1 answer
38 views

How to compare a single (counted) value to a mean for statistical significance in R

I am working on a project estimating salmon escapement through visual spawner surveys. We conducted weekly surveys and counted the number of individuals we observed. These numbers were used in an Area-...
T.Palmer95's user avatar
0 votes
1 answer
50 views

What does it mean for a variable to be statistically significant but not selected as important by classification methods?

I'm basically playing around with some lipidomics data to practice, so my question is purely theoretical. I wanted to see if I could find lipid classes that differ between two groups and I was ...
maglorismyspiritanimal's user avatar
1 vote
0 answers
28 views

Is it valid to do a Bayesian analysis on an old paper which didn't publish a Bayesian analysis? [closed]

Is Bayesian Analysis, and the judgement it allows on the chance of a true effect being significant despite an overall analysis being not statistically significant, something that can be ...
user356816's user avatar
0 votes
0 answers
8 views

Better Prediction model with large DIC value

I am using Integrated Nested Laplace Approximation (INLA) to predict birds population.In my (INLA) models, I've noticed that when I include auto spatial correlations, my predictions are good and make ...
Usman YousafZai's user avatar
0 votes
2 answers
34 views

How many z-scores should be used when determining a p-value?

Suppose one wants to test whether the average sum of tickets in a box is X. Since the z-score is computed by sampling N tickets and then seeing how many SE's away the average is from the expected ...
Andrei's user avatar
  • 1
0 votes
0 answers
32 views

Which test do I need for testing differences in two populations?

I would like to compare the different variables in a life table of two populations and check if the differences are significant. However, I am totally unsure which test is appropriate. For example, I ...
Octtta's user avatar
  • 1
0 votes
0 answers
13 views

How to compare two distributions of Coefficients of Variation?

For my master's thesis (machine learning) I am studying the effect of using a certain type of classifier on the stability of feature importance techniques (SHAP & LIME) under different levels of ...
Matteo Ballegeer's user avatar
1 vote
0 answers
11 views

How to Evaluate Interaction Effects in Propensity Score-Matched Samples:

Suppose I want to study the association between an exposure X and outcome Y, and I have used the propensity score to match each exposed subject with those unexposed but with similar characteristics ...
zjppdozen's user avatar
  • 347
1 vote
2 answers
69 views

Significant interaction when only one of the two main effects is significant? [duplicate]

I have run a mixed effect logistic regression model, and I have obtained a significant main effect of Factor A, while no significant main effect of Factor B. However, the interaction between Factor A ...
Katherine's user avatar
  • 115
1 vote
1 answer
22 views

What statistical analysis method should I use for two independent variables with interval data and a dependent variable with continuous data?

I'm doing a study on the potential of date leaf ash and date fruit seeds as replacement materials for cement and sand, respectively. For date leaf ash (DLA), there is 0% replacement and 5% replacement,...
TheBlueRail's user avatar
0 votes
0 answers
40 views

How to statistically test an increase?

I have two temporal networks in which I report their number of nodes and the average degree in different time intervals: time Nodes_Network_1 Nodes_Network_2 Average_degree_Network_1 ...
Jina's user avatar
  • 1
0 votes
0 answers
18 views

The difference between significant differences pased on positive and negative ranks on Wilcoxon signed ranked test [closed]

I am wondering about the interpretation for the results of Wilcoxon signed rank test. Now I am comparing between three predictions made by three different sets of feature and I want to test the ...
Eno Mo's user avatar
  • 1
2 votes
1 answer
39 views

Can a result be statistically significant if the null value is one of the 95% confidence intervals?

I'm reading a paper on the effect of 2 different drugs on mortality. The results are reported as RR 0.96 (0.94-1.00), p=0.03. The p value indicates significance but the upper 95% limit is 1.00. Am I ...
user356816's user avatar
4 votes
2 answers
156 views

Significant variable and very low R-squared [duplicate]

I'm testing if economic growth before an election is correlated with vote percentage in elections. So I have one independent variable in my model. The problem is: my independent variable is ...
Sina Alvandi's user avatar
1 vote
1 answer
21 views

How many samples should I test to be 95% sure that no error exists?

If I have a million population of products and I will tolerate no error in them. How many samples should I test to be 95% sure that no error exists? I am new to statistics. I know you might need some ...
Polime's user avatar
  • 11
0 votes
0 answers
9 views

Which variables should I include/exclude in my regression analysis for GDP? [duplicate]

I am wanting to establish the impact mobile money has on Kenyan economic growth. I am building my regression model and have collected data for GDP, consumption, government spending, investment, net ...
Georgia's user avatar
0 votes
0 answers
42 views

Statistical Tool to Compare Raw Data to Lower Resolution Data to Determine the Resolution When Data Is Lost

I have dimensional data collected from a product every 1 second. The product runs can often last days and analyzing the entire dataset can be difficult due to the large amount of datapoints. The ...
Chris's user avatar
  • 11
2 votes
0 answers
26 views

T-tests for differences of slopes of linear regression when intercept is forced through 0

I am looking to compare the differences in linear regression slopes between two datasets and at first I used a two sample t-test. However, when I force the intercept through $0$ for both regressions (...
Mitchell's user avatar
3 votes
2 answers
58 views

Effect of interaction term: what if one of the main effect is not statistically significant?

Suppose we have the following Poisson regression model: $\log(y) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2$ Where, for example, $y$ is the number of children in a family, $x_1$ is the ...
robertspierre's user avatar
0 votes
0 answers
23 views

Statistical significance between sigmoidal regression curves

I have a data-set of weight ~ temperature that was measured in 3 species (A-B-C), each consisting of 2 genotypes (a-b) that were grown in 3 soils (1-2-3) each. Weight ~ temperature for each Soil x ...
unknown's user avatar
  • 95
4 votes
1 answer
31 views

When testing significance between a regression coefficient of two data sets using a Z-test, how to interpret the z-value?

I am following the methods described in: Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, ...
jack kelly's user avatar
5 votes
1 answer
146 views

Statistical tests for data on occurrence of reptiles found under different types of artifical cover objects

I am studying small-bodied ground snakes and skinks (lizards). I use artifical cover objects to determine their presence. Artificial cover objects (hereafter CO) consisting of 7 different types of ...
BAE's user avatar
  • 53
0 votes
0 answers
35 views

Number of levels for ordinal DV, variance, and statistical significance

I am interested in how closely a person places themselves to a certain situation in the responses they give. The responses are coded, and I create a score, with the lowest value indicating the person ...
grey's user avatar
  • 663
3 votes
2 answers
85 views

How do I work out the significance of main effects in negative binomial models with more than two factors?

I am trying to look at how the number of events X is affected by the three factors A (4 levels), B (2 levels) and C (2 levels) using a negative binomial model as follows: ...
Insect_biologist's user avatar
2 votes
1 answer
62 views

Statistical test for binomial draws with unknown number of trials

I have two binomial random variables $X_1 = \text{Bin}(N, p_1)$ and $X_2 = \text{Bin}(N, p_2)$; same number of trials but different probabilities of success. Suppose that the number of trials $N$ is ...
lmyt's user avatar
  • 21
5 votes
2 answers
575 views

How does reducing my alpha-threshold to 0.005 affect my required sample size?

Among statisticians there is an ongoing debate whether the long-standing "standard" threshold for declaring the outcome of a statistical test as "significant" (a.k.a. alpha-level) ...
PuzzledBiologist's user avatar
0 votes
2 answers
88 views

Null hypothesis in clinical trial question

Consider a clinical trial where we want to study the evolution of the occurence of some event during two time periods, with and without treatment. We know that, not only a difference close to 0 won't ...
geaidc's user avatar
  • 1
0 votes
0 answers
10 views

How to compare peak location and tail length of two different distributions?

I have the distributions of the fraction of people in each income bracket in a town in 1990 and 2020. The total sample size is the same in both, and assume that the incomes have been adjusted to ...
SNIreaPER's user avatar
4 votes
2 answers
166 views

Which statistic to use for measurement between two samples [closed]

I am measuring stem widths of a control plant and a mutant plant and want to determine if there is any significant difference between them. The sample sizes are not the same. When I use a test like ...
Ruochan Liu's user avatar
3 votes
1 answer
24 views

Appropriate method for seeing how similar two items on a survey are?

I have a survey and we want to look at a pair of questions we think are very similar and possibly asking the same thing. If they are we'd like to reduce the two questions to one question. Both ...
sixfortyseven's user avatar
5 votes
3 answers
432 views

Hypothesis testing for detecting a (damped) sinusoidal signal in noise

I have a signal in white noise that has the following form: \begin{equation} r[t_i] = A e^{-t_i/\tau} \sin{(\omega t_i + \phi)} + n[t_i] \end{equation} I would like to test whether the signal (1st ...
P. Egli's user avatar
  • 161
0 votes
0 answers
28 views

Which test would I use in this situation?

I have a sample of people, $n=300$. All $300$ should be offered both of two therapeutic treatments (Treatment A and B). I will be collecting data on how many people were offered A only, B only, and A ...
Josh Sewell's user avatar
3 votes
1 answer
74 views

Remove non-significant independent variabels and re-run multiple regression

We have done a multiple regression with the 3 Theory of planned behaviour variables (subjective norm, perceived behavioural control and attitude) plus 2 added variables (because previous research ...
Carl Berglund's user avatar
0 votes
0 answers
20 views

Structural break test, individual regressors

Say I have an equation $y=\alpha_0+\alpha_1 x_1 + \alpha_2 x_2$ and I'm testing for a structural break (SB) in one of the slope coefficients. How would I discriminate between the breaks. For example ...
cel's user avatar
  • 11
5 votes
4 answers
941 views

Statistically significant difference

I am reading a medical paper, where it says: There was a statistically significant improvement in dyspnea in the IPC group at 6 months, with a mean difference in VAS score between the IPC group and ...
STRICKLAND_7's user avatar
0 votes
1 answer
47 views

Statistical testing on non-random sample?

I am working with a non-random sample in an observational study but would like to do statistical testing to show certain trends and processes. Statistical tests assume random sampling, which is not ...
manaTEA's user avatar
  • 103

1
2 3 4 5
128