New answers tagged statistical-significance
0
In your example, you would reject the first seven. The BH procedure is a step-up procedure: after ordering your pvalues as $p_1,\dots,p_m$, you should take the last $k$ such that $p_k \leq \frac{k}{m}\alpha$ regardless of whether you have some $p_{k'}$'s with $k'<k$ that do not satisfy this.
Maybe someone else can do a better job of providing intuition, ...
3
That test is to see if the means of two samples are different from each other. So what you really tested is if the average wind speed is different from the average number of passing yards.
If you want to see the effect of windspeed on passing yards, what I would suggest is building a linear regression model, where wind speed is the independent variable, ...
1
Want to punish all criminals? Throw everyone in jail. Never want to punish an innocent defendant? Don’t ever prosecute anyone.
As you require a higher standard of proof ($\alpha$ or specificity), it’s harder to convict. If you need fingerprints, DNA, and two eyewitness, that’s a lot of evidence compared to just relying on one eyewitness and some prints. You ...
1
Both an ANOVA and mixed modeling approach will work for your data. I am more familiar with mixed models, and can speak better to that. However, you are correct that you can run your analysis as a mixed model and then get an ANOVA table afterward. If you are using R, then you can use lmer to first estimate the mixed model:
require(lme4)
m <- lmer(outcome ~...
3
This notation is describing a statistical test that was applied in order to determine whether or not there is a "statistically significant" association between two variables. Here, the specific question is whether "content related to losing fat or weight" is associated with what site you are on (Thinsipration vs. Fitspiration). The null (default) hypothesis ...
0
Given that the response in the control group are independent of the one in the experimental group you could try to fit just a single model. And you include control/experimental as a variable and include the interaction with the 20 predictors and the newly added variable. Significant interaction would mean difference in effect of the predictor between the ...
2
If I understand you correctly, you are asking about testing the significance of the difference between two regressions on the same data set. This is not a standard significance testing problem. A standard significance testing problem states a null hypothesis ($H_0$) about how the data were generated, and uses the distribution assuming this $H_0$ of a certain ...
6
I have many questions about the proposed approach, mostly because it is not clearly described. It's unlikely this question can be edited to give us much insight into the setting and rationale for this approach. Nonetheless, a few points can be addressed.
Sums of variances obtained by randomly partitioning a dataset 30-fold cannot be called "robustness". A ...
1
Estimating a regression without a constant term is essentially saying "I know the intercept of my model is equal to zero. I don't need to estimate it because I know with certainty what it is." That's a pretty bold claim to make. If you're right, then you can estimate the slope with slightly increased precision. If you're wrong, your estimate of the slope ...
1
So here is my answer. This is by no means the best answer. Rather it is just a basic general framework that is used.
When I fit a model I typically like to decide the "best fit" by splitting my data into a training set and a validation set. The training set is used to actually fit the model and the validation set is used to test the model's accuracy. The ...
4
If you want to provide evidence for equivalence (by contrast with evidence for difference), you can perform a t-test for equivalence using TOST. (In the below "$\theta$" is the difference between groups you are estimating a la $\bar{\mu}_{1} - \bar{\mu}_{2}$ or whatever.)
General 'negativist' null hypothesis: $H_{0}^{-}: |\theta| \ge \delta$, with $H_{\text{...
0
Your research question is whether user experience differs between android apps and progressive web apps. You designed an experiment and collected data. Your results turned out not significant at the a priori chosen $\alpha$ level of $0.05$. Assuming the data collection and all statistical assumptions were met, the p-value gives you the probability of ...
1
At a basic level, you would certainly have to account for the number of men and women in the pool at each time when a selection is made. Do that for each selection. Then you can calculate (for the 2 year period) the number of men and women that would have been selected if the choice was random.
Suppose there were only two selections (to keep it simple). At ...
answered Jan 11 at 13:49
1
What tests generally do is that they tell you whether what you have observed is unlikely under the null hypothesis, which is a probability model for random data. Note that this does not mean that you can only use them if the data are indeed random. If you apply a statistical test to nonrandom data and you get a significant result, it means that your data ...
1
The problem with Frequent Pattern Mining is that, in practice, one is likely to discover a titanic number of patterns with many redundancies. Many researchers have devoted substantial effort to come up with clever ways to improve the quality of pattern discovery, i.e. bring forth the most interesting patterns and bring down redundancy.
The word '...
3
1) Are all my independent variables X1,X2...Xn called as exposure variables?
The term "exposure variable" comes mainly from the epidemiology and causal inference literature. It is not completely well defined, but usually there is one (main) exposure, an outcome, and the other variables are either competing exposures, or confounders (some could be mediators ...
1
Yes, if that is your alternative hypothesis:
I have data in which I have a clinical suspicion that the mean of x is greater than the mean of y
then you can use a one-tailed test. If it is going to be part of a paper for publication then you should perhaps consult the journal's guidelines since it might require a two-sided test. In any case, just be very ...
1
The chi-square ignores the ordering in your categories. It will respond to any kind of association between the variables.
The Spearman takes account of the ordering but is responsive to a tendency for monotonic association (when both variables tend to be larger together and smaller together, or when both variables tend to move in opposite directions)
[The ...
answered Jan 9 at 1:39
Glen_b -Reinstate Monica
231k25 gold badges467 silver badges818 bronze badges
0
The easiest is to use a computer package such as SPSS or STATA to evaluate the significance, I don't think you can easily use tables for this.
If you are a programmer you can use the permutation test (which I think is implemented in some computer packages). With observations $(x_1,y_1),\ldots(x_n,y_n)$, randomly generate $m$ permutations $(y_{(1)},\ldots,...
1
I think it'll be difficult to perfectly model this without some domain knowledge, but a permutation test might provide a pretty decent approximation.
In general, a reasonable approach is to apply a hypothesis test. Here the null hypothesis is: the teams are evenly matched. Now to apply the null hypothesis testing methodology, we need to pick a test ...
4
You need to decide what tests you need to make, because although the models are the same, the tests that the software automatically conducts will differ.
To see why, consider the simplified version of the situation you originally proposed, where there are two regressors $x_1, x_2$ and their interaction $x_1x_2$. Let $\xi_i$ be the corresponding ...
3
The way that it is presented is quite counterintuitive, as it is shown as a set of intervals even though it looks like a list.
However, it means that:
*** is used for values BETWEEN 0 and 0.001
** is used for values BETWEEN 0.001 and 0.01
* is used for values BETWEEN 0.01 and 0.05
. is used for values BETWEEN 0.05 and 0.1
empty character is used for ...
0
To build a predictive model, you must select the most relevant features in the model else if you have large number of features then your model will not converge. So will not get the right results from the model.
As you rightly mention that if features are highly correlated then the variables coefficients will be inflated.
For predictive model my suggestion ...
4
Well, in order to say if there is a statistical significance you need to have some kind of distribution of the results, and this will depend on the "league". I.e. the distribution of results in the NBA is different from the one of the Youth League I used to play in :)
If you don't have a theoretical distribution for the results, you can try to use available ...
2
While it cannot create the table in exactly how you specified, you can calculate risk ratios (and other measures) using the zEpid library. This library supports both calculating from summary counts (details here) and directly from pandas DataFrame objects (details here).
The library does not directly calculate p-values, but you can easily do this by a ...
2
Actually, Genetic Algorithm based approaches, can also give you the optimum list of features besides giving you the ranking. I suspect that the specific library you are using is simply giving you the feature ranking.
I would suggest to use an Optimization algorithm and code up the objective function(rmse or r2) yourself and perform optimization. The ...
1
I'll assume that grades are measured on a 0-100 scales, and that there are no edge effects to be worried about (that is, the majority of students do not score near 0 or 100 in their classes).
To investigate the effect (here, I should say effect or association since this is not a causal model, but I digress) of MCAT score on med school performance, you can ...
4
I would like to add one point to EdM's answer, that has not been mentioned yet.
Statistically significant but not important
This could be some random feature of the data and because of the multiple testing problem some features are significant in the dataset purely by sampling.
However, it could also be that the overall effect of an explanatory variable ...
1
There is a lot to unpack here, so I'll just answer a few of these.
Confounding occurs when a predictor and the outcome share a common cause. Usually, the presence of unadjusted confounding yields a biased estimate for the relationship between the predictor and the outcome. If you are building a predictive model, you don't need to think about confounding ...
11
A few general points before answering the individual questions.
First, in logistic regression (unlike in linear regression) coefficient estimates will be biased if you omit any predictor associated with outcome whether or not it is correlated with the included predictors. This page gives an analytic demonstration for the related probit regression.
Second, ...
6
6) Can you let me know what is the problem here and how can I address this?
With all due respect, by reading your post I see only red flags due to misapplication and misunderstanding of the statistical methods. I would suggest employing a statistician (and at the very least, reading a great deal on clinical prediction models/regression modeling from Frank ...
1
No, it doesn't mean that. The model with the intercept is usually uninteresting and whether the intercept is significant isn't important. I don't think there's a sensible way of making the intercept significant, but there's no reason to do so.
So, you can add independent variables.
answered Jan 1 at 11:51
7
Significance testing consists of defining a rejection region, and rejecting if the data is in that region. The size of the region is its $\alpha$ value. If two different regions are different shapes, then even if one is smaller than the other, there can be places that are inside the smaller one but not in the larger one.
Dave’s answer explains that KS tests ...
9
The F-test specifically examines for differences in variance and does not need to be sensitive to other differences such as mean. KS has to be sensitive to every kind of distributional difference, whether that difference is mean, variance, or multimodality.
Think of the F-test as a specialist that will be great at finding differences in variance at the ...
2
You are tackling the problem the wrong way.
First you have a hypothesis.
Was your hypothesis was that all questions were related to your effect of interest?
If that's true, then the level of significance for this test is the maximum p-value you found.
Therefore it is not significant.
If your question was that at least one question was related to the ...
0
Pearson Correlation and Spearman Rank Correlation coefficients could be used to measure correlation with human judgements for different algorithms.
1
A non-statistically solid rule used in epidemiology is called "The 10% rule". It states that when the Odds Ratio (OR) changes by 10% or more upon including a confounder in your model, the confounder must be controlled for by leaving it in the model. If a 10% change in OR is not observed, you can remove the variable from your model, as it does not need to be ...
2
The term confounder (or confounding) is used quite loosely these days. Usually, what people mean is to include additional variables into the regression model. In the medical literature, these are often age or gender. I'm not sure how this relates to interactions, which are merely used to allow for non-linear effects in a linear regression model. Consider the ...
3
In general, I agree with the following statements in the editorial Moving to a World Beyond "p < 0.05" which is part of the special issue Statistical Inference in the 21st Century: A World Beyond p < 0.05 of The American Statistician:
What you will NOT find in this issue is one solution that majestically replaces the outsized role that statistical ...
1
First of all, a graph represents the trends in your sample but not in your population. Also, I assume that the number of the controls is greater than the number of idividuals with ASD. Thus they have greater variance and there could be some influencial points that make the difference.
You should first find if there are any influential points in your ...
2
$H_0$: Amount of missed shot is geometrically distributed with $p = 0.7$
$H_1$: Amount of missed shot is not geometrically distributed with $p = 0.7$
\begin{array} {|r|r|}\hline \text{amount of missed shots} & 0 & 1 & 2 & >2 \\ \hline \text{empirical frequency} & 72 & 18 & 8 & 2 \\ \hline \text{expected frequency} &...
2
If we know the null hypothesis is not exactly true, yet the result is not statistically significant, then that is an issue of sample size, or statistical power. Statistical significance is not really a goal, it's a necessity that one achieves with appropriate statistical power. Given the same effect size, the results of two experiments can be statistically ...
6
I agree with the answer by Peter Flom, but would like to add an additional point on the use of the term "significance" in statistical hypothesis testing. Most hypothesis tests of interest in statistics have a null hypothesis that posits a zero value for some "effect" and an alternative hypothesis that posits a non-zero (or positive, or negative) value for ...
1
You can just state the result: "On average, Gurples were 10 cm taller than Cheebles (Difference in Height = 10 [5, 14]; mean, 95% CI, p=0.03)."
8
"statistical significance" sounds official and formal. It is a way to put a stamp of "validity" on your result, which is not afforded by the procedure. Probably that's why ASA is against the term.
In my opinion one of more honest yet non-technical phrasing would be something like:
The obtained result is surprising/unexpected (p = 0.03) under the ...
26
I don't think the objection is to just the term "statistically significant" but to the abuse of the whole concept of statistical significance testing and to the misinterpretation of results that are (or are not) statistically significant.
In particular, look at these six statements:
P-values can indicate how incompatible the data are with a specified ...
answered Dec 24 '19 at 13:58
0
In case you aren't aware, the p-values returned for each category you are tested assess if the individual dummy is significantly different than your determined reference group. I believe you are looking for a global test to assess if the variable itself is significant. In this case you would use a "chunk test" AKA a "general linear F-test".
See this thread:...
Top 50 recent answers are included
