Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.
0
votes
0answers
16 views
Statistical Significance test for difference in FScore
For a classification task, I have developed two methods and FScore(harmonic mean of precision and recall) of both the classes serves as the performance criteria. How can I check whether the difference ...
1
vote
1answer
44 views
Can I use a regression model with ANOVA significance greater than 0.05?
I have a multiple regression using SPSS. The significance of my model in the ANOVA table is p=0.174 which is >0.05. what does this mean for my model? Can I still use it and proceed in the ...
4
votes
1answer
32 views
How to rank order performance of four systems using aggregates of pairwise assessments of system performance?
I am currently designing an experiment to evaluate 4 iterations of a system. As it is rather difficult for a human to judge the output of the system by assigning it a score or several scores, I plan ...
1
vote
2answers
79 views
Stats: Relationship between Alpha and Beta
My question has to do with the relationship between alpha and beta and their definitions in statistics.
alpha = type I error rate = significance level under consideration that the NULL hypothesis is ...
2
votes
1answer
25 views
Statistical tool(s) to correlate perceptions with Likert-type scales
In relation to perceptions of teachers, I want to measure:
perception of helpfulness
required level of helpfulness for the respondent to be satisfied, and
how important helpfulness is (as an aspect ...
0
votes
1answer
28 views
Simple Mixed Model with 1 Fixed and 1 Random Effect
I have various datasets I need to analyse regarding soil properties, all in the same fashion, with one fixed effect (which is a position along a transect, indicating different land uses). Now my main ...
1
vote
0answers
14 views
Significance testing for a group of samples
For instance, I have a bar chart - in which I have 4 samples/bars. for these 4 bars, I want to check statistical significance for all combinations.
How can I do it? Can unpaired t-test be used - to ...
1
vote
2answers
59 views
Correlation not significant because there is not enough variance?
I have a question about correlations again.
I have a dichotomous variable that I want to correlate with the another one (metric) by using the point-biserial correlation coefficient. I get a ...
3
votes
2answers
77 views
How to compare percentages for two categories from one sample?
I want to compare two percentages on a single categorical variable from one sample.
For example, in my data set, a variable can take on two values (i.e. A and B). Of 586 instances, 87.4% is in ...
3
votes
1answer
20 views
Statistical significance and winner of multivalent test
This should be a fairly straight forward, but I can't seem to figure out the answer. I'm doing some A/B(C/D/E...) testing on a website and measuring impressions and clicks. What method should I be ...
0
votes
2answers
62 views
Pearson correlation is not significant although effect is .30 What could be the reason?
I have a correlation between two variables of 0.30, hence a medium effect. but it is not statistically significant and I want to report the reasons for that. looking at the test statistic I see that ...
4
votes
3answers
115 views
Problem with p-values from A/B testing
I'm pretty confused right now and the more I look at it, the more confused I get. Overthinking!
I'm doing some A/B tests for my website. I'm using the significance calculator that's in this article:
...
2
votes
0answers
13 views
How to test significance of pre-post difference scores on 8 measures (1 group)
I have a set of pre and post treatment scores on 8 measures. I'd like to test which of these show significant improvement after treatment. If I use within-subjects t tests it will mean carrying out 8 ...
2
votes
2answers
191 views
1
vote
0answers
32 views
Two-way anova with interaction term: what is the point of a post-hoc test?
I have a statistics interpretation question. I've recently performed a two-way anova to identify an interaction term between my categorical independent variables (genotype + temperature) that ...
2
votes
1answer
67 views
How to compare two datasets using metrics drawn from unknown distributions and with small sample sizes?
I have two datasets consisting of metrics from several experiments. Dataset 1 is the collection of results of experiments E performed by user A on product A, repeated N times. Dataset 2 is the ...
1
vote
1answer
112 views
Looking for help with interpreting my OLS models
I have been busy building an explainatory regression model for a companies' social behavior. In the table below you find the variables, ordered by theme. The ...
3
votes
0answers
146 views
Ratios of means - statistical comparison test using Fieller's theorem?
I would really appreciate any suggestions with the following data analysis issue. Please read till the end as the problem at first may appear trivial, but after much researching, I assure you it is ...
1
vote
0answers
19 views
Test regression parameter against a constant in SPSS
This is a pretty basic question, but I can't find an answer by searching for different statements of the same problem.
Is there a straightforward way to test if a regression parameter is different ...
0
votes
0answers
31 views
Correcting Classical Linear Regression Model (CLRM) violation
How would you correct for violated error term assumption of CLRM in stata. If our hypothesis is correct, our assumption of error term is violated. I am trying to correct this on stata but don't know ...
3
votes
1answer
31 views
How to prove the number of distinct distributions in a group of distributions?
Let's say we had 5 distributions: A,B,C,D,E.
An ANOVA test would tell us whether or not all of the means are equal, and thus a low p-value would mean at least one of the means is unlikely to be ...
1
vote
0answers
28 views
Hidden Markov model & statistical significance
I am using HMM to explore the language development of one individual with six variables as input, which are trained into three sequences.
In the first state, the covariance of variable A and B is ...
0
votes
1answer
64 views
Interpretation of Spearman correlation for small sample
I am currently being confused by different opinions regarding the spearman correlation interpretation. Some says, $\rho<0.2$ can be ignored as small/weak relationship, some says $\rho<0.1$, for ...
0
votes
0answers
42 views
Statistical test for multiple comparison with dependent observations (MATLAB)
I want to use matlab to perform a statistical test to find if there's a significantly higher mean return in any of n different sample populations. However, I can only find post hoc (although they can ...
2
votes
1answer
74 views
Test for significance of correlation matrix
If one wishes to test if the correlations in a correlation matrix are statistically significant as a whole group, one can perform a likelihood ratio test of the hypothesis that the correlation matrix ...
0
votes
0answers
26 views
How to test if two populations are different with the x axis a time variable
I have two populations: Class A and Class B. Both populations have two variables: ...
0
votes
0answers
26 views
What kind of statistical analysis to use for a homework paper? Anova?
I am writing a paper where I have to pretend to have conducted an experiment. Each of the experimental groups and control has different people.
There are 8 experimental groups and 1 control.
1a 1b ...
0
votes
0answers
25 views
Calculating differences between groups and across time (age)
Let's say I have data of one variable $V$ measured for two groups of individuals ($a$ and $b$) and I want to see whether there is a significant difference between them and over time, i.e. $V_a$ stays ...
1
vote
0answers
60 views
Expected value of sums of squares of a blocking factor in a split plot design
How can we show that in a split plot design, the expected value of the sum-of-squares of the blocking factor has expected value equal to $k\sigma^2_{\text{block}} + \sigma^2_{\text{error}}$?
I tried ...
-1
votes
0answers
26 views
Why is an ANOVA interaction no longer significant after including a covariate? [duplicate]
I have initially run a 2x2x2 Anova and found some significant interaction between the independent variables. I then ran an ANCOVA where I added a covariant. I am confused with how to interpret the ...
0
votes
0answers
4 views
Resolution of a Test Method
How does one calculate the resolution of a test method, that is, the smallest difference between two measurements, also called the resolving power or discrimination?
0
votes
1answer
43 views
Conservative confidence interval
One month before the election, a poll of a large number of randomly selected voters showed 65% planning to vote for a certain candidate. The newspaper article reported a 90% conservative margin of ...
0
votes
0answers
31 views
Repeated 100x 10-fold cross validation, what is the sample size when doing an significance test?
I iterated my 10-fold cross validation 100 times for several methods. Now I want to use a t-test to test if the results are significant. However, I'm not sure what the sample size is.
Is the sample ...
1
vote
0answers
27 views
Statistical analysis of multiple algorithms over a single dataset
I have a dataset $X = \{x_1, ..., x_n\}$. I also have three algorithms, $A_1, A_2, A_3$, that each take a single data point as input and produce some measure of how well they performed. If I apply ...
0
votes
1answer
18 views
Statistics test for concentration of extract effecting germination
I am undertaking biology course work and am a little stuck.
My experimental hypothesis is: There is a significant correlation between increasing tomato extract concentrations and reduced seed ...
0
votes
0answers
45 views
How do I test for significant difference between sets of time ordered data?
I have data about activity on a game site over a period of time (~1.5 years). I want to compare a sub-population's activity between certain time segments so that I can say things like "Group A played ...
1
vote
0answers
19 views
Deciding if my mutant strains are significantly different from my wild type from data measured over time or from gradients
I have a set of data which is the oxygen consumption of a wild type (WT) and a few mutants. I averaged the data and plotted graphs with each mutant and WT series and plotted a line of best fit. Is ...
1
vote
2answers
76 views
How to compare results of a questionnaire consisting of 40 likert items, before and after an intervention?
I need some help!
In my research (medical education field), I had 32 participants answering a questionnaire consisting of 40 likert items rating their level of confidence in different aspects of ...
5
votes
1answer
83 views
Speed up web a/b tests with sample size checkpoints
Before starting an a/b test with a control and one experimental route, I can calculate a required sample size based on conversion rate estimates for both routes. I can get a good estimate of the ...
0
votes
1answer
65 views
Overall rank from multiple ranked lists
I've looked through a lot of literature available online, including this forum without any luck and hoping someone can help a statistical issue I currently face:
I have 5 lists of of ranked data, ...
0
votes
0answers
36 views
How do I prove that my results are significantly better in this situation?
I have a test suit for an optimization problem with 270 problem instances. For all these instances I have a lower bound and a single solution of another algorithm. I do not own this algorithm and am ...
0
votes
0answers
22 views
Difference between chi statistical methods (pearson/likelyhood/linear-by-linear)
Can someone briefly explain the differences between these three tests? When would you use one over the other? Which tests are associated with a test of independence, homogeneity, or goodness-of-fit?
...
1
vote
1answer
38 views
Chi squared test of homogeneity
Suppose you want to test the null hypothesis:
H0: p1 = 0.65, p2 = 0.27, p3 = 0.01, p4 = 0.05, p5 = 0.02
to determine whether a population's distribution matches the proposed proportions. You ...
0
votes
3answers
51 views
Area to the right or left when computing test statistic
Can someone generally explain when the area to the right or area to the left is sufficient in solving a statistical problem (using z, t, or chi)? For example, in a chi-squared problem, if we're ...
1
vote
1answer
67 views
Multiple hypothesis testing and F tests?
Does the multiple hypothesis testing problem apply to the calculation of an F statistic for joint significance? It seems to me that the more variables you are including in your test for joint ...
0
votes
0answers
46 views
One way ANOVA and Duncan's test
I'm working on a program for statistical calculations.Always have two sets of data...
I calculated all the other values from ANOVA and Duncan's test:
SS Effect,df effect,MS Effect,SS Error,df ...
0
votes
0answers
31 views
Whats the biggest difference in calculating a simple regression model with/without a constant term?
I have calulated an OLS with and without a constant term. However, besides that the values are different I haven`t really found anything valueable?
Therefore my question is:
Whats the biggest ...
0
votes
2answers
26 views
Hypothesis / Statsitical significance between continous variable and dependent binary variable
Sorry I am a stats newbie, I understood that I can use the search feature, I tried to search but I am afraid that I am not using the right terms and the results returned are not quite relevant to my ...
0
votes
0answers
34 views
Evaluate Quantile Regression results and t-stats at different levels.
I have a few questions regarding quantile regression and how to interpret the results. I have several independent variables and I want to figure out which combination of variables have the highest ...
0
votes
1answer
41 views
Finding value of residual
Consider the LSR line: y = 1158.86 - 5.54x
A researcher sought to examine the effect of average teacher salary on the average (total) SAT score. The data, measured at the state level so 50 ...
