Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

1
vote
1answer
14 views

How are degrees of freedom used in the Welch's t-test to determine p-value?

I'm trying to understand using Welch's t-statistic. I'd like to compute the t-score between two distributions with different numbers of samples and different variances. The Null hypothesis is that ...
0
votes
0answers
10 views

I have 3 gene datasets, 2 genes are sig in all 3, what is the probability of this being by chance?

I have three large RNA-seq datasets, two genes were found to be significant in all three datasets, what is the probability of this happening by chance? not sure what info i would need so here is ...
0
votes
0answers
14 views

Different set of predictors significant for different sample sizes - how to interpret results?

So I am trying a GARCH framework with external regressor(s) to predict returns. The external regressor, $y$, intuitively has useful lags that could predict the response. I'm slowly accumulating data ...
0
votes
0answers
10 views

Comparing adjusted odds ratios in a model

I have a pretty basic question. Say I run a logistic regression model with 2 variables both dichotomous. falls= sex+stroke+depression. I get 3 different odds ratios, one for males vs females, one for ...
0
votes
0answers
11 views

Comparing a Z-score from a model fit on real data against a randomized input

I have a quite complex statistical model that fits the parameter $X$ for some input data. The form of this model is irrelevant for the further question (at least I think so). Let's say I provide two ...
0
votes
0answers
11 views

Best Test for Limited Data

I'm just getting my start in learning statistics, and I'd love to be able to apply some of the tools I'm learning to my professional work. Sometimes I find it hard to determine how to apply that ...
0
votes
1answer
41 views
+50

How to calculate conditional hazard at time zero

I am looking for a statistical method to calculate the conditional hazard before the diagnoses or study has started; in other words, time is not a variable. For example, here is a hypothetical ...
53
votes
9answers
8k views

What does “Scientists rise up against statistical significance” mean? (Comment in Nature)

The title of the Comment in Nature Scientists rise up against statistical significance begins with: Valentin Amrhein, Sander Greenland, Blake McShane and more than 800 signatories call for an end ...
3
votes
1answer
22 views

Can I perform a McNemar's test with only proportion and denominator information?

I have the following count and denominator information. The 630 subjects are the same people over two different periods of time. Can I conduct a McNemar's test to compare the dependent proportion ...
0
votes
0answers
14 views

Determine interval which deviates from random distribution?

I have a distribution of values extracted from an original dataset: X = [0.02, 0.05, 0.1, 0.12, 0.14, 0.15, 0.16, 0.17, 0.17, 0.18, 0.2, 0.3, 0.4, 0.6, 0.8]; I can do the same procedure to extract ...
0
votes
0answers
5 views

Comparing scores and financial data for incentive and non incentive locations

I am doing an analysis which is considering two groups: one group with incentives (test) and one group without incentives (control). I want to see if scores (i.e. recommend, overall satisfaction, ...
0
votes
0answers
11 views

False Discovery Rate (FDR) one-stage and two-stage method

Which is the difference between the classical one-stage and the two-stage method? which one should I report?
0
votes
0answers
18 views

Goodness of fit test for any regression model?

Is there a general goodness-of-fit test for any kind of regression model? My problem is that I have a deep neural network that tries to predict some real value labels using high-dimensional input. The ...
5
votes
3answers
149 views
+200

A/B testing ratio of sums

Context Consider the following scenario for a company selling goods online. A user can purchase several items (i.e. basket of items), some of which are of particular importance and are tracked ...
0
votes
1answer
44 views

Asymmetric data analyses

I have a data set containing bacterial count values before treatment and after treatment. These data look asymmetric. Please advise me for a suitable statistical test to find if the introduction of ...
0
votes
0answers
35 views

How do I set up the data for a Friedman Test?

Below is a small sample of a very large data set (over 10,000 records). All I'm trying to do is determine if there's a statistically significant difference found in the ordering of supplies among ...
17
votes
2answers
1k views

categorizing a variable turns it from insignificant to significant

I have a numeric variable which turns out not significant in a multivariate logistic regression model. However, when I categorize it into groups, suddenly it becomes significant. This is very counter-...
3
votes
3answers
99 views
+50

Describing Specialization

I am trying to formalize an observed trend. To simplify what I am trying to do, suppose a dataset of salesmen selling items A, B, C, D. In the year 1950, the proportion of each sold is (0.2, 0.2, 0.1, ...
0
votes
1answer
27 views

what is alternative/opposite to piecemeal statistical approach?

I have only basic knowledge about bio-statistics. For not normally distributed data, I used Kruskal-Wallis test to investigate the statistical significance between different variables. I performed an ...
0
votes
0answers
5 views

Differences in means of raster files

I am analysing long-term data on a raster. I group this data in three parts depending on a factor value. Now, I am taking means in each group pixel by pixel and would like to detect pixels on the map ...
0
votes
1answer
34 views

Question about statistical test for classification problem in a very unique case!

I'm working on a non-standard dataset with deep learning, and I want to prove that my method is better. We have a dataset composed from dataset A and dataset B where each one is composed of ...
0
votes
0answers
10 views

How can I use CCA to compare Datasets of Different Dimensionality? [closed]

I'm working on a class research project and I'm using CCA to compare different activation patterns in a neural network. I have the same number of data points for each comparison however the number of ...
0
votes
1answer
33 views

How important is a statistically significant intercept?

I've created the following model: log(consumption) = a + b*log(GDP) + c*log(GDP(-1)) + d*log(consumption(-1)) The slope coefficients are all statistically ...
0
votes
1answer
32 views

Interpret the slope coefficient

The slope coefficient for SO2 is quite small (0.33), especially as compared to the other three slopes). Does this suggest that the effect of SO2 is therefore not very important?
0
votes
0answers
13 views

How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
-1
votes
0answers
12 views

Comparing two classifier with non-normal accuracy distribution

I'm working on a non-standard dataset with deep learning, and I want to prove that my method is better by comparing accuracies of my classifier and that of a baseline. I used ANOVA and Wilcoxon but I'...
0
votes
0answers
93 views
+50

Logistic regression including instrumental variable (“ivprobit” in R) has coefficients with much reduced significance

I am trying to run a logistic regression including instrumental variables by using "ivprobit" function in R from the package called "ivprobit". If I do not include the instrumental variable, the "glm"...
2
votes
2answers
178 views

Justifying the trend component in a time series?

I am working on a time series forecasting problem ,I used Dickey–Fuller test to check the stationary of the time series and the p value is 0.000835 , hense I rejected the null hypothesis and assumed ...
1
vote
1answer
34 views

How to use classifier a to test whether the boundary between classes (and the classes, themselves) is (are) similar across two datasets

I have two datasets which each contain two classes (four classes total). I suspect that both datasets contain the same kinds of classes (i.e. the boundary that distinguishes classes is similar across ...
0
votes
0answers
9 views

Pearson and Spearman correlation on absolute values with its change over time

I am trying to correlate the absolute value of one variable with its change over time. I have 4 time points in my data when the same subject is re-assess: baseline, TP1, TP2 and TP3. It seems that ...
0
votes
2answers
23 views

Clustering data multiple times to get statistics

Suppose I have some data (objects described by a number of variables like diameter, etc...) and that I want to run K-means clustering on it. Suppose now that I run the algorithm multiple times (1) on ...
0
votes
0answers
56 views

Can a Wilcoxon signed rank test be applied on dated data?

I have data for military male and female orders for types of clothing purchased from 2012 to 2018 (below is a very simple example - most of the data is one item purchases). If I look at one type of ...
0
votes
1answer
18 views

odd insignificant results using pairwise.perm.manova

I am trying to complete pairwise comparisons of 9 forest communities. Communities were significant at p < 0.001 using adonis. However, when I use the same bray curtis matrix with the pairwise.perm....
0
votes
0answers
13 views

Appropriate visualization and statistical testing using resampling stats

I'm using a linear SVM to perform classification on a dataset. It seems to me that there are many ways to visualize my results and report their statistical significance, and I'm unsure of best ...
0
votes
1answer
28 views

comparsion of treatment groups based on different controls

I have experimental results from two days where I wasnt able to keep the same conditions from day1 to day2, so I have two sets of data: control day1 -- 2x treatment groups day1 control day2 -- 2x ...
2
votes
1answer
51 views

How many data points to see if one model is significantly better than other models in a finite set?

I am working on a linear regression problem, let's say I'm looking for a model that predicts housing prices as a function of number of rooms and square feet. Assume for each attempt to price a house, ...
0
votes
0answers
14 views

How to calculate the significance of a pairwise comparison distribution?

I have a variable ($X$) that depends on two parameters (say $P$ and $Q$), each of which can take six values each ($P_1\cdots P_6$ and $Q_1\cdots Q_6$). I take pairwise values of each parameter like $...
1
vote
2answers
27 views

confidence interval of function of parameter

Self Study Question: We already know that the confidence interval of a parameter M is (-1,2). We are supposed to find the confidence interval of 1/M. The function is 1/X, which is One to One, so ...
0
votes
0answers
17 views

Wald test for Poisson distributions

This question asks: $N_A$ and $N_B$ are variables of the counts of the number of events 'A' and events 'B' respectively. Those variables follow Poisson distributions with parameters $\lambda_A$...
-1
votes
1answer
20 views

Calculate the statistical significance of survey questions

I have a question: How are you ? and two possible answers: Good and Bad. If for example I got over 30 questions ,10 Good answers and 20 Bad answers, which methods I should use to check if the ...
0
votes
2answers
26 views

which condition from question to select for which hypothesis test?

I'm learning statistics and having hard times understanding hypothesis. Suppose we are testing the hypothesis that sugar is linked to obesity. Which will be for what type of hypothesis. for example- ...
0
votes
0answers
35 views

How can I use k-fold cross-validation to determine whether a linear regression model performs significantly better than chance?

I have an experiment in which I present a subject with 1820 inputs. For each input, a response is produced in ~25,000 separate output variables. I have a function F which produces a feature vector ...
2
votes
1answer
49 views

Significance Tests - Why Would a Low Sample Size Cause Problems?

Okay, on my AP Statistics test on the chapter on significance tests, the following free response question appeared: When the manufacturing process is working properly, NeverREady batteries have ...
0
votes
0answers
9 views

How to accurately measure conversion after a click

I analyze very large sets of site performance data for advertising and or product recommendations. By and large, I measure: Impressions = 100,000 Clicks = 1000 (direct result from impression) ...
0
votes
0answers
11 views

test of significance for anomaly detection

I am using an anomaly detection algorithm (Twitter's Anomaly Detection) that uses SH-ESD method. I 'm finding anomalies in a time-series of the daily counts. I can obtain TP, FP and FN measures. Is ...
0
votes
1answer
10 views

What is the frequency for each level of categorical variables to obtain the most reliable results?

I'm working with a real dataset. In some variables, the frequency between each of the variables is not uniform. For example, in the Occup variable, the frequency for levels 2, 3, and 6 is very small. ...
0
votes
2answers
19 views

Test statistical significance with small number - Fisher's exact test

Before the part of the question related to the title, here is some background: I'm build a model that tries to predict a binary outcome ($Y$), given a set of features. Some of this features are ...
0
votes
0answers
14 views

Minimum sample size calculation a posteriori

I am running a number of experiments where I obtain a large number of observations with one variable (yielding two sets of values; many more than would be needed for a hypothesis test). I want to test ...
1
vote
1answer
10 views

appropriate test for comparison between two variables

I have two variables , Variable1 and Variable 2. Variable 1 has three levels { Pizza, Pasta, Breadstick } Variable 2 has five levels { American, German, Spanish, Japanese, Chinese } The response ...
2
votes
0answers
22 views

Interpreting the margin of error when it is larger than the sampled proportion

Say that I have a population of $N=500$ individuals, and I'm sampling $n=60$ of them for a certain characteristic which is expected to be quite rare (say green eyes). In my sample I find out there are ...