Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

Filter by
Sorted by
Tagged with
1
vote
0answers
9 views

A priori publication of statistical analysis plan (SAP) for clinical trials

This is somewhat of a regulatory question, rather than statistical question. But I assume that the readership of this community will be the most knowledgeable on this matter. Question: Is there any ...
3
votes
0answers
18 views

Guideline one using which statistical test?

Are there any guide lines on determining which test to use? For example, given 100 subjects with both Exam A and Exam B, and some student observations for those exams, i want to compare which exam is ...
2
votes
1answer
11 views

Looking for scientific validation for a heuristic model

The problem: infer the nationality of a person from a limited number of features (name, email, ...). I do not have enough "ground truth" to use ML techniques, I'd like to try what for a ...
0
votes
1answer
23 views

Use of Chi-Square test of independence / Fisher Exact Test?

One of my research hypotheses is that individuals from Southeast Asia who are ethnically Chinese are more likely to experience racially motivated hate crimes than their counterparts from other ethnic ...
1
vote
2answers
38 views

How to perform a statistical significance test between 2 groups of binary data?

I have a binary column where 0 means fail and 1 means success. This column have been grouped by a second column called events. Here's a data sample before grouping: ...
0
votes
0answers
6 views

Which one to use: Cliff's Delta or Vargha-Delaney's A12 to know the effect size?

I have to compare the predictions of two regression models (both are predicting on same dataset, but one is using entire data and another one is using only pruned data). To know whether there mean ...
0
votes
1answer
23 views

Box and whisker plot looks significant but paper says it is not?

I am reading this paper "Long-term vitamin D treatment decreases human uterine leiomyoma size in a xenograft animal model" and one of their plots looks strange to me. According to the paper,...
0
votes
0answers
6 views

Case weighted classification in Python

I'm trying to find a good method for showing whether or not some variables are statistically significant explainers of a binary (0, 1) response in Python. Catch is, I'd like to weight the data points ...
1
vote
0answers
6 views

Panel Match: ATT estimate level of significance

I am a novice user in R. I'm working with Version 1.3.1093 and Windows . I'm working on a Panel Data Set for all activist interventions by hedge funds in Europe for the period 2005 - 2019(time ...
0
votes
0answers
13 views

A very big JB value in Jarque-Bera test [duplicate]

I have run the JB test for my data using two different commands. I am quite clueless about what conclusion I can make from the second picture which shows results of 0. This tells me that there is no ...
0
votes
0answers
16 views

Am I using the right test to to evaluate my treatment effect?

I have calculated a p-value (with this solution), for the percentage change from the control to treatment group (1 compared to 2, 3 compared to 4 etc.) below, for each level of ...
0
votes
0answers
18 views

Hypothesis Testing for non parametric data

Suppose that for a website page, I would like to monitor the completeness of all the page request data I received, a percentage number. (ie. For all the requests we receive, how many of them are empty ...
0
votes
0answers
13 views

p-value and choice of variables to enter the model

I would like to ask you what is the best way to select variables for helping the model to improve. The task consists of spam detection. I selected some features that I think they might be useful for ...
0
votes
2answers
16 views

Rewriting coefficients of a regression

We are interested in the variable $\alpha_1$, in the equation $ y_{iv1}=\alpha_0 + \alpha_1group_v+x_ia_x+ \gamma y_{iv0} + \epsilon_{iv1} $ Where $y_{iv1}$ is a dummy variable, and $x_ia_x$ denotes ...
0
votes
1answer
38 views

Does central limit theorem help in making inference only about the population mean and not other parameters?

As per the weak law of large numbers, if your sample size is large, your mean of the sample is likely to be closer to the population mean than in a smaller sample. Additionally the CLT tells us the ...
0
votes
0answers
17 views

Maximizing the difference between conditional distributions

I am running an A/B test. For each of the two tests, I get a set of samples $x_1,x_2,\dots$ each associated with features $f_1,f_2,\dots$. When I look at the distribution of values $x_1,x_2,\dots$ for ...
0
votes
0answers
30 views

Asymptotic variance of a sample skewness in Bai and Ng (2005)

I am reading a paper from Bai and NG (2005) and trying to code asymptotic variance of sample skewness from Theorem 1: Suppose $X_t$ is stationary up to sixth order. Then $$\sqrt T (\hat \tau - \tau) \...
0
votes
0answers
13 views

Can I perform One-way ANOVA after transormations and division?

I'm wet scientinst and not that good at math so sorry in advance. I'm performing ELISA, and after that I'm getting absorbtion and my calibration curve in order to convert absorbtion to pg/ml. (Pic1) ...
0
votes
0answers
45 views

Hypothesis test for detecting level at which players most likely to fail [closed]

I'm new to hypothesis testing. I have a data with ~50000 sample with player_id, date, level(number), status and session. event_datetime: The date and time at which the event was received (Local time) ...
0
votes
0answers
14 views

Finding cross-correlation p-value for non-normal time series with CLT

I have two non-normally distributed time series with 77 data points (each) which I am cross-correlating. The time series were pre-whitened to remove autocorrelation. Since I am not trying to use ...
0
votes
0answers
19 views

How to show in a scientific paper format the post-hoc test of 4 variables involved, when 3 by 3 variables are significant?

In a multi-factorial experiment involving 4 variables (species(2x), temperature(2x), nutrients(2x) and light(3x)), 3 by 3 variables, for example, "species", "temperature" and "...
0
votes
0answers
16 views

Is sample skewness statistically significant?

I've been searching for a research that would examine statistical significance of a sample skewness between two financial time series of returns. Unfortunately, I was not able to find any good or ...
1
vote
1answer
12 views

Are significant small effects (standardized coefficients) valid on extremely large samples?

I am testing a path model on a sample of 12.000 subjects. All the path coefficients have extremely low values (e.g. 0.003, 0.012) yet they prove significant. How do I know I am detecting a ...
0
votes
0answers
13 views

Event study: test for cumulative abnormal returns

I am performing an event study, across different firms, using the market model theory. $$\ {R_{it}=α_i+β_{i}R_{mt}+ε_{it}}$$ With Abnormal Returns (AR):` $$AR_{it}=R_{it}−α_{i}−β_{i}R_{m t}$$ How can ...
0
votes
0answers
8 views

What are some plausible criteria for starting a statistical meta-analysis process?

Before stating the problem, let me indicate my background so the community can understand my capabilities, or lack thereof. I am a trained biologist. However, during my bachelor's I studied many CS ...
0
votes
0answers
9 views

Statistical test for two sets of word embedding vectors being 'similar enough'

I have two models which, for given input query q, output 'k' similar sentences from the same corpus. The i-th element of the output is a vector-embedding representation of the i-th most similar ...
0
votes
0answers
7 views

How do i find if a shift in Mean is significant in two Datasets with uneven rows?

I have Two datasets with Productivity values for Two different Quarters in a year(q1,q2). Now I want to identify if the shift in mean From q1 to q2 is significant between these two datasets. Will my ...
2
votes
1answer
32 views

Conflict between OR confidence interval and p-value

I'm currently reading this study about the adherence to health checkups in Austria. Table 2 displays some Odds Ratios for attending health checkups between groups of different educational statuses. I ...
1
vote
0answers
13 views

Computing statistical Significance of Rank Order Question

In my questionnaire, I have a rank order question concerning three different videos. Each respondent must rank each of the videos either on position 1, 2 or 3. All respondents are in the same group (...
0
votes
0answers
8 views

How to compare two weighted linear models with that are not nested

Below is one of the datasets used, the second has an identical Age column but different NumAge values. The NumAge is the ln() of the number of observations at each age. I created a weighted (weighted ...
0
votes
1answer
29 views

Which one-sample statistical test for ordinal variables?

I have one group of <25 subjects and multiple evaluation scores: New images of every patients has been rated by a radiologist compared to old images (1 to 5 with 1 Markedly worse, 3 similar and 5 ...
0
votes
0answers
14 views

Mixed models in SPSS

We are investigating the migration patterns of a surgical prosthesis. We do this by taking X-rays and noting the migration along the 3 orthogonal axes, where T0 is the state immediately after ...
0
votes
2answers
27 views

How to compare predicted Y against a constant at different levels of a continuous X

I have a simple linear regression with a continuous $X$ (Likert scale, range 1 to 7) and a continuous $Y$ (people's prediction about an event, range 0 to 100). I want to see if the predicted $Y$ at ...
0
votes
1answer
23 views

Statistical test that compare observed events with expeected

Could anyone enlighten me on what will be the appropriate statistics to test whether a series of events is significantly different from the expected or not. For example, there is a pool of infinitive ...
0
votes
0answers
14 views

How to test difference between two subgroups in one sample - how to decide about normality in the subgroups?

I distributed one questionnaire to a group of 191 respondents (Questions on Likert scale). I split the sample into men a women. I would like to see, if the difference between them is significatn (if ...
0
votes
1answer
22 views

Contrast post-hoc test on non-significant factor levels

Is it meaningful to interpret the contrast result from a post-hoc test on factor levels that have non-significant coefficients in the fitted mixed model regression? I'm using a contrast test using <...
1
vote
0answers
18 views

Whitney Mann U test vs Kolmogorov Smirnov for a highly skewed continuous variable

I would like to compare task time to completion distributions (measured in minutes) across two groups. The distributions are highly skewed (the means are 300 to 400 times larger than the medians) and ...
0
votes
0answers
23 views

2d data statistical significance

In my multiclass classification problem, I have several clusters that intersect with each other on a 2d graph (formed after dimensional reduction using linear discriminant analysis). What statistical ...
0
votes
0answers
6 views

How to test the equality of two proportions with multiple samplings?

I am searching for a statistical test to compare two proportions of responsive cells. I know that Khi2, z-test and Fisher’s exact test allow this kind of comparison but the issue I have is that I have ...
1
vote
0answers
25 views

Testing if a day of data is an outlier

I have a time series which exhibits a relatively similar behavior from one day to another. I want to know if a specific day of the time series is an outlier in the sense that it significantly differs ...
0
votes
1answer
6 views

statistical testing of proportions to include multiple experiments

I am struggling with understanding what is the correct statistical method to use to determine the statistical difference in proportions for multiple experiments. Let me explain my scenario. I have run ...
0
votes
0answers
18 views

My boss ask me to test for a change in frequency distribution over time in panel customer data? [closed]

For each day of the month I have a number of clients (around 10k per day) that ask for service. They are categorized by "Quality" using a score based on their personal informations: Bronze ...
0
votes
0answers
15 views

Which statistical test should I use for ranking percentages between groups?

In an experiment, there are multiple candidates, each candidates took a series of tests, represent in percentage. However, not all candidates took all tests. Assuming each test is weighted equally. I ...
1
vote
0answers
32 views

Statistically significant difference in R^2 between two linear regression models

We are interested in if the difference in R^2 between two linear regression models is statistically significant (the metric of model performance doesn't necessarily have to be R^2 but that is what we ...
2
votes
1answer
45 views

Does the problem of multiple testing also apply to the testing of assumptions?

When dealing with statistical tests, sometimes we can run into cases where many assumptions would apply and consequently would need to be tested. In complex models, testing many assumptions at 5% may ...
2
votes
2answers
69 views

test if 5% difference is significant

I have a dataset that contains 20000 chess matches. According to these matches, white won 50% of the games while black won 45%. I want to test if this 5% difference is significant. How can I do that? ...
0
votes
1answer
25 views

Dealing with results from thousands of regression models

I have a dataset containing 100 observations and 4 independent variables. These 100 observations can be categorised into 30 groups. In other words, for any given iteration, one observation per group ...
0
votes
0answers
19 views

Comparison of Coefficients of Variation (CVs)

I have to compare the variability of measures between 2 groups (geographical regions). Each group consists of a limited number of elements (i.e., the number of districts of each region); hence, a ...
0
votes
0answers
8 views

Interaction term not significant but post-hoc tests are

I'm running a GEE model in SAS with an interaction term between two categorical variables group2 and lbl. Running a joint test, ...
0
votes
0answers
5 views

Descriptive comparison of interactions

I want to identify trends and analyze several discrete variables (around 100)(value= 0 or 1) between 3 independent samples. Each random sample has 10000 observations. For example, subset 1 is 10000 ...

1
2 3 4 5
105