Questions tagged [statistical-significance]

Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).

Filter by
Sorted by
Tagged with
0 votes
1 answer
17 views

Statistical test for correctness of expectation

First time posting here. Suppose I have a random variable $X$ and after various algebraic rewriting and calculus I have arrived at a closed form of $\phi = \mathbb{E}[X]$. However, I am not totally ...
  • 3
2 votes
1 answer
15 views

Measuring events in sales time series

I am trying to measure the effect of an event on some sales data. An event might be turning on a new payment method for customers, or a discount coupon promotion. Does it make sense to try a time ...
0 votes
0 answers
14 views

Dealing with zero measurments regarding statistical tests

Let say that I have two sets of measurements (two groups of patients A and B, 30 people each). I used some technique that checks particular gene expression. The measurements range from 0 to 100 with ...
  • 55
1 vote
1 answer
46 views

Confused to interpret p-values

Sorry, if question is meaningless but I have started to learn concept of P-value and confused to interpret that, if actual results are in favour of hypothesis but p-value are against it then what ...
  • 111
2 votes
2 answers
49 views

What does it mean when a variable is significant as a binary variable but not as an ordinal variable?

Let's say I am running Cox proportional regression model for a variable with 4 levels (strongly agree, agree, disagree, strongly disagree). When I code it as an ordinal variable with these 4 ...
  • 21
0 votes
0 answers
9 views

How to check if variation of intercepts and slopes between random effect is significant (linear mixed model)?

I have conducted a linear mixed effect regression for the day of green-up in the Arctic. Regions are random effects, weather variables are kept as fixed effects. My data, after scaling to center, is ...
  • 354
1 vote
1 answer
34 views

Statistical test for difference in means using only group mean and sample size

I have aggregated survey data for various groups, but do not have access to the raw responses. Is there an appropriate test for statistical significance between group means with only access to the ...
1 vote
1 answer
12 views

Why same t-score for contrast pairs involve continuous x continuous interaction (emtrends)?

I want to calculate the difference in simple slope estimates for two-way interactions that involve two continuous variables. But I found the t-score is all the same for the contrast-pair results when ...
0 votes
3 answers
88 views
+150

On the usage of many train/test split ratios

Is there any benefit in evaluating model performance under different train/test ratios? To me, this sounds kind of nonsense because of these reasons: I cannot access such a parameter when the model ...
0 votes
0 answers
23 views

Too many statistically significant results with propensity score weighting

I'm currently performing a drug-wide associatinon study (DWAS) in a population-based cohort, in which I'm iterating over commonly used drugs to assess whether they have an association with my outcome ...
0 votes
0 answers
6 views

Compare Debt Collection Liquidation Rate between two Collection agencies - Choice of statistical test

I am looking for the right statistical test to compared the performance between two collection agencies base on liquidation rate. Collections Liquidation Rate measures the dollar amount successfully ...
  • 1
0 votes
0 answers
23 views

How to calculate the hypothesis testing of lognormal distribution?

everyone. Since, my sample data is right skewed, so I take log on it and it is normal distribution. It is Y=log(X), Y is normal distribution. And I want to test if X = 30. I set null hypothesis as X = ...
0 votes
0 answers
11 views

Minimax hypothesis testing - prove equality of the empirical cost

For the general minimax test, when the Bayes decision rule is designed assuming the prior is $q$, prove the following equality of the empirical cost $$ \left.\frac{\mathrm{d} \varphi\left(\hat{H}_p, p\...
0 votes
0 answers
24 views

Likert scale statistical analysis

I have 2 groups that have completed several Likert (1-5) surveys and I want to test if there is a significant difference in the responses between the 2 groups for each survey. However the number of ...
0 votes
1 answer
39 views

One of the main effects not significant, but interaction term significant

A is significant B is not significant A x B is significant Do I say that (1) There exists a ...
  • 145
0 votes
1 answer
12 views

Checking interaction between one dependent continuous variable and two independent continuous variable

I am trying to figure out if there is a way that we can perform some statistical test to check the interaction between two independent continuous variables and a dependent variable in R. I have three ...
0 votes
1 answer
34 views

Which degrees of freedom estimate to use for two-tailed t-test?

I have two samples: Sample A = businesses that have not received a grant Sample B = businesses that have received a grant Ua = population mean of survival rate for businesses in A Ub = population mean ...
  • 1
0 votes
0 answers
19 views

Binning: Selection of Range (Cutoff) for a Variable

In a current modeling work, we only use facilities with the secured portion % in the range of 70-100% (the range can be from 0-200%). These facilities (in the 70-100% secured percent range) are used ...
0 votes
0 answers
31 views

Wilcoxon test for unrelated samples

I have a cancer dataset with two groups of patients, those who responded to the therapy and those who did not. I run cell type enrichment analysis, to get an idea of the cellular structure for each ...
0 votes
0 answers
8 views

Knockoff filters simple explanation and importance!

Knockoff filters are new in the field of variable selection. Can someone provide (or refer to) a slightly simple understanding of the topic? Also, what is the fuzz about this new method compared to ...
0 votes
0 answers
38 views

Model diagnostic plots for anova test

Could you please help me in interpreting the model diagnostic plots? I feel there is a problem with the scale-location plot?
0 votes
0 answers
26 views

Calculate minimum sample size needed for statistical significance

I am implementing a market survey question and would like to know the minimum sample size needed to report the results with 95% statistical significance. Not sure if this is relevant, but here is a ...
1 vote
0 answers
9 views

Which test to use for associations between categorical independent and ordinal outcome

I am analyzing factors that influence program retention in a medical program. Our dependent variable, retention time, is split into 4 categories (<1 month, 1-3 months, 3.1-6 months and >6 months)...
1 vote
0 answers
22 views

Multicollinearity Market mix modeling

I want to know what can be the best approach to handle multicollinearity. I am building a regression model with just 4 independent but all important variables and am not able to control the VIF. ...
  • 21
1 vote
1 answer
33 views

Calculating an effect of intervention on two groups with very different values

Imagine I run a marketing campaign in two cities with very different number of citizens (like A: 1000 and B: 1000 000). Purchases_A and Purchases_B define the number of products sold in each city: <...
0 votes
0 answers
25 views

Are my samples not independent?

I want to predict soccer games and want to calculate the statistical significance of my features for selection. One of my features is the amount of points both teams have before the game. Does this ...
  • 13
0 votes
0 answers
31 views

Statistical tests vs model selection

I'm analyzing health data with case reports which contain patient information including diagnosis, age, sex, location etc. The sample sizes are not very large. I want to explore the data for ...
  • 1
0 votes
1 answer
18 views

Should I use standardised or unstandardised beta coefficient in this scenario?

I am running a regression on Tax payment and investment. Tax payment is numerical and investment is categorical like, number of projects carried out. When I run the regression the a unstandardised ...
1 vote
2 answers
30 views

CV for Model selection - Why is significance testing not needed?

Say we have a model and some hyper-parameter with L values, and our goal is model selection. A k-fold CV outputs L accuracies (each accuracy is an average over K values). The best model corresponds to ...
  • 995
0 votes
0 answers
17 views

Statistically compare two algorithm outputs

I want to test the efficiency of two different algorithms. I have two random samples from the same population. On one sample, I run Algorithm X; on another sample, I run Algorithm Y. There are 30 ...
  • 1
0 votes
0 answers
25 views

Johnson-Neyman plots for glmer models (logistic, mixed effects) in R against non-zero values

Note: this is a different question from the one I asked here, but I'm using the same example model. I have an R glmer (logisitc mixed-effects) model that looks ...
  • 23
1 vote
1 answer
37 views

Can I take the p-value from a probability table for a modified z-score, just like a z-score?

I have a question regarding the modified Z-score, can I use this score in a Z-probability table, i.e. the p-value from a modified z-score? where: z-score = (xi - μ)/σ (the z-score probability can be ...
0 votes
0 answers
22 views

Does Mann-Whitney U test require equal variance or not? [duplicate]

I have already read that the test requires and also that the test does not require equal variances of the samples tested. For example: Requires: https://www.isixsigma.com/tools-templates/hypothesis-...
0 votes
2 answers
48 views

How can I back transform a log data to interpret t-test and get original CI?

I have data that is not normally distributed. I can log-transform it to be normally distributed, and then perform a t-test and get confidence intervals (CI). But how do I interpret the results of the ...
  • 11
7 votes
3 answers
471 views

How do you determine if there is a significant relationship between two variables with several factors affecting it, using R?

I want to check if there is a significant relationship between x and y, and then check if this relationship is affected by different stimuli (2 levels), colonies (2 levels), and lightings situation (2 ...
  • 71
0 votes
1 answer
36 views

How does center variable help address multicollinearity problems for interaction with polynomial terms

I hope someone can help me better understand how center variables can help address the problem of multicollinearity problem when the regression includes interactions with polynomial terms. My ...
0 votes
2 answers
52 views

effect vs significance

I've seen on some plots e.g. with values for the statistical of significance p = 0.7 and for the effect size of g = 0.5. Is it possible to say there is a certain effect (size) when it is not ...
  • 2,441
0 votes
0 answers
17 views

What does the CI interval mean? [duplicate]

I created a plot with the help of ggbetweenstats: I wonder about the meaning of the CI-interval which is [-0.37, 1.49]. What does this mean? Sorry for cutting the labels on the y-axis. The numbers ...
  • 2,441
2 votes
2 answers
26 views

Is there an iterative sampling strategy for finding the strata with the highest average yield?

I am planning a research project to determine which is the optimal strategy to isolate as many unique bacteriophages as possible for a particular bacteria. In short the process goes something like ...
1 vote
1 answer
28 views

Johnson-Neyman plots for glmer models (logistic, mixed effects) in R

I have an R glmer (logisitc mixed-effects) model that looks something like: ...
  • 23
0 votes
0 answers
13 views

How to interpret Shap summary plot on causal application with X,T,Y?

I am trying to make sense of how to interpret the following Shap plot given the context of a causal model. See article of relevance: https://towardsdatascience.com/causal-machine-learning-for-...
1 vote
1 answer
23 views

How to test statistical significance of standard deviation?

I have two lists of values A and B. I calculated the standard deviation of A and the standard deviation of B. The standard deviation of A is bigger than that of B. How can I test if the difference in ...
  • 31
0 votes
0 answers
18 views

Statistical Test of the significance of difference between 2 ordered columns

I'm trying to come up with an appropriate statistical test for the significance of the difference between 2 ordered columns A = [a0, a1, ..., aN] and B = [b0, b1, ..., bN]. Each of the columns is of ...
0 votes
0 answers
21 views

How to determine if difference is significantly large enough

I've a data: name team score score_diff a x 10 10 a y 20 10 b x 400 10 b y 390 10 c x 400 390 c y 10 390 How to know, for each value in name, whether the ...
0 votes
1 answer
23 views

Is there a test that can tell me, even for low counts, the probability that the obtained distribution of values eg from a die is truly random?

The background is that I have data and a number of independent variables. For each variable I want to ascertain the probability that the distribution of that variable's values is random for that data. ...
  • 103
0 votes
0 answers
13 views

Comparison of samples of optimal objective function values of a deterministic optimisation problem solved by a stochastic optimisation solver

Motivating Problem Assume that I must pack tins of soup into a box, where the box has a fixed volume. I can choose the shape of the soup tin, and the volume of the soup tin, and the packing ...
0 votes
1 answer
35 views

Anova with stratified Cox model

I'd like to use anova to investigate which variables are important to my Cox model outcome. I tried using anova from ...
  • 335
0 votes
0 answers
31 views

Robust standard errors in Cox survival analysis

The final step of running a Cox model I've been working on involves performing log-likelihood ratio tests to check the significance of each predictor to the model. I'd like to do this using ...
  • 335
0 votes
0 answers
22 views

Overall effect for non-binary categorical variable, Cox

There are many great answers on Cross-Validated explaining that the p-values for cox.zph (or other regression models) in R refer ...
  • 335
3 votes
1 answer
38 views

Which metric(s) can I use to evaluate how well a group of binary models agree in their predictions?

My request is best explained with an example. Suppose the upcoming week of (American) football matches are ...
  • 1,862

1
2 3 4 5
118