Questions tagged [statistical-significance]

Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How can I perform evaluate how well a model fits data that spans many orders of (tiny) magnitude?

For some dataset, I am fitting a model and then looking to evaluate how well this model 'fits' the data. The data in most cases, will range several orders of tiny magnitude, pictured below my data ...
user avatar
  • 1
1 vote
0 answers
24 views

Issue with bounded in probability

I have tried to prove the following problem that I read in the lecture and it seems not transparent to me. Suppose that $Y_{i}$ be independent random variables (with $i=1,2,3,...$). Each has the ...
user avatar
3 votes
1 answer
59 views

Usefulness of KS tests and other similar distribution comparing tests

I am working on a machine learning binary classification problem. I have an outcome variable status called as loan paid and <...
user avatar
  • 1,758
0 votes
1 answer
25 views

How many values should one validate to achieve a desired confidence?

If I have a set of 100 values which have been programmatically calculated and I would like to validate those calculations by performing them manually, how many manual calculations would I need to do ...
user avatar
  • 3
0 votes
0 answers
17 views

Statistical test when comparing oversampling to no oversampling on ANN

I use 70% of the dataset for training and 30% for testing. I use oversampling on the training dataset with an ANN. I use the test dataset on my ANN and look at the performance of oversampling against ...
user avatar
0 votes
2 answers
35 views

Is confounding a source of Type I errors?

I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
user avatar
  • 31
2 votes
1 answer
248 views

Unusally high F value ANOVA

I am running a one-way ANOVA to compare a numeric continuous variable between four groups, and getting a very high F-value. Below are the descriptives and code I used in R (I get similar values when ...
user avatar
0 votes
0 answers
47 views

Alternatives to linear model regression

Let's say I have a univariate linear regression model LMR in which ...
user avatar
0 votes
1 answer
15 views

How to compare variables with data from multiple experiments?

I have an experiment in which i compare the amount of attached bacteria in three different conditions. I have repeated this experimented three times, so now i have three datasets with the same data (...
user avatar
1 vote
0 answers
19 views

How to test a product of a OLS estimators?

I estimated a vector $\beta $ using OLS $\hat \beta =(\hat e_1 \hat e_2 \hat e_3)$ and I have the covariance matrice $\hat v(\hat \beta )$. How can I test this null hypothesis $H_0 : \hat e_1 \hat ...
user avatar
  • 177
0 votes
0 answers
8 views

How to measure the change in customer flow from brand A to brand B at two points in time (pretest and posttest) after a stimulus?

I am conducting a research in which I have data on the purchase of 4 brands of some consumers in a first period of time (pretest). But I also have the data for the purchase of those 4 brands and the ...
user avatar
  • 1
6 votes
2 answers
275 views

How to report a P-value?

I am writing the results of my thesis, and I have a few questions on how to report P-values. When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-...
user avatar
  • 91
1 vote
1 answer
47 views

Which statistics test should I use?

Group High is a year in which there was a high temperature change and group Low is a year in which there was a low temperature change. The classes represent an the percentage of coverage for a species ...
user avatar
  • 11
0 votes
0 answers
27 views

How to test the statistical significance of several clustering algorithms? [closed]

I have computed ARI, NMI and Silhouette scores of several clustering algorithms on single cell RNA sequence data. These metrics are computed three times for each algorithm and for each dataset. Now, I ...
user avatar
0 votes
0 answers
28 views

How can I check if the hit rate of the correct answer significantly differed between groups?

I have a task in my experiment, where participants had to select an emotion out of 7 options dependent on a video they saw. The participants were split between three groups. Now I want to check, if ...
user avatar
1 vote
1 answer
21 views

How do I test that two samples don’t differ significantly by their demographics?

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
user avatar
  • 125
0 votes
0 answers
9 views

Establish convergence to normal area

I will try and make my question abstract, since I have two problems of the same overall type. I am given a time series $Y_t$, t=1,...,T. I know that this time series has a (slowly) changing ...
user avatar
  • 21
0 votes
0 answers
21 views

Using the same subjects in different measures

I'm working on an experiment where I collect 3 categories of reaction time data form subjects who are video game players, distinguished by the game that they play (eg. RTS, FPS, MOBA). I wish to ...
user avatar
3 votes
1 answer
29 views

Why do discrete choice models (such as MNL) not require test set?

The central challenge of Machine learning models is perform well on unseen data. The data is randomly split into train and test set. The test set acts as surrogate for unseen data and is used to ...
user avatar
  • 141
0 votes
0 answers
10 views

Simplified DID: How to run significance test

If we know that Variable C = Variable A - Variable B, and Variable A is the difference in two groups (binary data) during time period X. It is statistically significant. Variable B is the difference ...
user avatar
0 votes
0 answers
15 views

Meaning of endpoints of ONE confidence interval

From wiki, we should understand the confidence interval by: in large repeated samples, 95% of the realized confidence intervals (computed from samples) can cover the true value of the population ...
user avatar
0 votes
0 answers
6 views

Multilevel models significance

For example, in the OLS regression (not multilevel), we have R^2 and p-value for F test. This p value indicates whether R^2 is significance or not. In multilevel models, we have R^2 as well (marginal/...
user avatar
  • 109
0 votes
0 answers
27 views

Comparing current and new version of Autoencoder

I am building an LSTM Autoencoder (unsupervised model) to detect anomalies in a time series dataset. The input is telemetry data from routers and I want to detect anomalies in the throughout of ...
user avatar
0 votes
0 answers
11 views

Back-transformation of confidence intervals

I have read that I can simply back-transform my confidence intervals from my (mixed) linear models, which seems very handy for model interpretation. What I don't understand is why log-transforming the ...
user avatar
  • 1
0 votes
1 answer
39 views

Statistical analysis (comparison) of time course experiments

I have data sets that represent multiple measurements over time. The data sets come from biological experiments in which we measure something in N individual cells, with around 250 measurements over a ...
user avatar
  • 1
0 votes
1 answer
30 views

Correlations significant at individual level, but not at group level?

For my research I have to do correlation tests to see how different measures of animal welfare are related to each other. When i run the test using the data at individual level, it shows a weak ...
user avatar
  • 91
1 vote
0 answers
18 views

Best way to compare income of two groups of people over time

I am planning a research study where, among other things, I want to understand how an intervention changes income in two groups. I plan to collect income data from each individual at three time points....
user avatar
  • 45
0 votes
1 answer
33 views

References: t-test and Chi-squared test can be conducted with unequal sample sizes

I have two independent groups: sample size of group 1 is 24000, and sample size of group 2 is 246. With the drastically different sample sizes in the two group, I need to use the Student’s t-test or ...
user avatar
  • 1
1 vote
0 answers
19 views

Question about confidence interval and standard errors on graphs

So I made a graph in R, it’s a linear regression of the same sample at different concentrations. It has a confidence interval built into it of 95%. Then I had to determine the concentration of an ...
user avatar
  • 11
1 vote
0 answers
18 views

What statistical analysis should I use [duplicate]

I have been measuring flux of CO2 in (mmol m-2 d-1) from two different rivers and I now want to do a statistical analysis using IBM SPSS but Im not sure where to start. One lake is showing negative ...
user avatar
2 votes
0 answers
14 views

Statistical test for temporal cross validation

I estimated the performance of my forecasting model and that of a baseline on 10 folds using temporal cross-validation. With which test do I assess if my model is significantly better than the ...
user avatar
  • 423
1 vote
1 answer
48 views

Collinearity problem

Consider a linear regression of this type: height: beta_0 + beta_1*weight. Adding BMI as parameter would add complexity to the problem or just cause a collinearity problem?
user avatar
0 votes
0 answers
18 views

What are the differences between "exact" and "inexact" tests?

What are the differences between exact and inexact tests? (i.e., McNemar's Exact Test vs McNemar's Inexact Test) When should each be used?
user avatar
0 votes
0 answers
12 views

My regression becomes insignificant when I add time fixed effects? [duplicate]

I have observations on a policy index and foreign direct investment for 30 countries, for 2001 and 2010. I want to explore the relationship between the two variables. I run an OLS regression and find ...
user avatar
  • 1
0 votes
0 answers
39 views

how to find standard deviation of sampling distribution without knowing population parameter σ

Standard deviation of sampling distribution is given as σ/sqrt(n). But usually the population parameter σ will be unknown. In that case, how do we calcuate the standard deviation of sampling ...
user avatar
  • 101
0 votes
0 answers
13 views

R statistical test for two groups with three categorial variables [duplicate]

I have two groups, in which three possible outcomes appeared at different frequencies, and I wanna know, if there is a significant difference in-between the groups. For instance, I have two potatoe ...
user avatar
0 votes
0 answers
22 views

R statistical test for two groups with categorial data

I have two groups, in which three possible outcomes appeared at different frequencies, and I wanna know, if there is a significant difference in-between the groups. For instance, I have two potatoe ...
user avatar
1 vote
1 answer
25 views

Simple question on statistical comparison of grand/pooled means

My problem: I am comparing the means of means of means in my data set. However, my stats software (GraphPad Prism) doesn't know that these said individual values (the means of means of means) are ...
user avatar
1 vote
0 answers
25 views

why the sum of two variables is insignificant while each of them are?

When I regress stock returns of period $t$ on stock ownership of period $t-1$, the coefficient on this lagged stock ownership is insignificant. However, when I disaggregate the lagged stock ownership ...
user avatar
  • 11
0 votes
0 answers
17 views

The exact distribution of the conditional distribution of the OLS estimator

This is the problem that I have tried figuring it out for a while, and I still need some advice because there is no explicit derivation in the textbook that I have seen so far. The problem looks easy ...
user avatar
1 vote
1 answer
40 views

Is a huge F-statistic OK?

I ran a multivariate regression model in R and got an overall F-statistic of 3,525.690 and three stars attached to it. This seems to be quite good from a statistical point of view. However, should I ...
user avatar
1 vote
1 answer
21 views

difference in proportion in multiple sets

This is the dummy example that I made up. If I want to know whether the boxer uses different proportions of punch based on opponents, what kind of test should I use. In other words, how can I tell ...
user avatar
0 votes
0 answers
11 views

Which alternative analysis to a 2-factor ANOVA when having 2 independent variables and one dependent?

So, I have a dataset where I need to test the relationship between 2 independent qualitative variables and one dependent quantitative variable. I wanted to use 2-way ANOVA but my independent variable ...
user avatar
1 vote
0 answers
25 views

how to compare the prediction of two models using statistical significant test python

basically I have the age prediction from one model: ...
user avatar
0 votes
0 answers
12 views

How to do hypothesis testing with real-valued data

Disclaimer: I don't mean real-valued data in the sense that the data could be any number in $\mathbb{R}$, I mean it rather in that there is no proportion/percentage. Let $A$ be a dataset that has $n$ ...
user avatar
  • 155
1 vote
0 answers
20 views

Interpreting fisher’s Z transformation and correlations

Part of my planned analysis in a paper is testing the difference in strength of correlations between depression and other things like social relationships and quality of life. This test is set up with ...
user avatar
0 votes
0 answers
25 views

How to compare two methods using regression

In the above question, I am supposed to compare method 1 (pre-use current) to method 2 (post-use current), using linear regression, I know you can use hypotheses testing to check if the difference ...
user avatar
0 votes
0 answers
11 views

Formally quantifying and assessing difference in calibration scores among different estimators?

Given a number of estimators $f_{1}, \ldots, f_{n}$ and a number of corresponding calibration scores $s_{1}, \ldots, s_{n}$ for each data group $g_{1}, \ldots, g_{m}$ with $m \ll n$. We would like to ...
user avatar
0 votes
0 answers
7 views

all variables at one of my level is not significant

I am modelling a logistic regression multilevel with 3 levels. Level 1 is individual, level 2 is household characteristic, and level 3 is regional characteristics. The results of the random effect ...
user avatar
1 vote
1 answer
33 views

How to read ggforest and significances for the cox model?

I run ggforest on a cox model and I wonder now, how I have interpret the output: Here, several factors of the variable country are shown. I stumble over the fact, that only countries with very few ...
user avatar
  • 2,187

1
2 3 4 5
115