Hot answers tagged statistical-significance
5
In addition to @gung's answer, I'll try to provide an example of what the anova function actually tests. I hope this enables you to decide what tests are appropriate for the hypotheses you are interested in testing.
Let's assume that you have an outcome $y$ and 3 predictor variables: $x_{1}$, $x_{2}$, and $x_{3}$. Now, if your logistic regression model ...
5
A test procedure goes like this:
(1) Define the sample space: 1024 outcomes of tossing a coin 10 times
(2) State the null hypothesis: A fair coin; i.e. $\mathsf{H}$ & $\mathsf{T}$ equiprobable, tosses independent
(3) Define a test statistic: You can use the sum of heads, or the number of runs, or whatever you like
(4) Perform the experiment & ...
4
I don't think this is anything to do with P-values. In any case you nowhere specify what test you have in mind.
The usual definition of a fair coin I take to be that heads and tails are equally probable, but nothing rules out "fairness" being a vague concept that can be made precise in several ways. In practice one should also be wary of -- to take one ...
4
Not exactly an answer but perhaps can help start a discussion.
To me the design of the questions seems problematic. The pre-question measures "determination" and the post-question measures "experience." Someone may be too anxious to decide to be focused, but then experienced a very focused session. I am not sure what you can get out of that. It feels like ...
3
There are two problems with your first approach: Multiple testing and the interpretation of the difference between significant and non-significant.
First, when you perform several tests, you increase the overall error rate. If you set $\alpha = .05$, you will still reject the null hypothesis 5% of the time when it is actually true (i.e. when there is no ...
3
If your goal is to see which methods are better than the baseline, then method 1 is correct. If your goal is to see which methods are better than each other, then method 2 is correct. Method 2 with Dunnett's test could also be used, this makes corrections for the multiple comparisons involved. However, the question of whether you need to make these ...
3
This is akin to a stepwise procedure and generally not recommended. Adjusting a model is good for prediction or for explorative purposes but it biases p-values if those are computed afterwards on the same dataset. Ideally, if the main objective is testing then you should specify the model in advance and stick to that.
You also need to take into account the ...
3
Solution 1 has an issue with sequence length. For instance, if you are interested in the word A and all your sequences have length 10,000 it is extremely likely that they all contain the word of interest in which case the Fisher test will not report anything significant, even if the occurrence of the word varies a lot within the sequences.
Solution 2 ...
3
If the number of observations is reasonably large and the distribution of responses not too jammed up one end or the other, a plain two-sample t-test may be reasonable, especially if you would be prepared to add such an item to similar Likert-scale items to produce a scale, were additional items available (this illustrates preparedness to assume that the ...
3
The intercept-only model is the 'null' model here, and the p-value associated with the model overall is given in the last line of your output.
You may notice it's the same as for the slope coefficient; that's because it's being compared to an intercept-only model.
You should definitely not regard including the intercept as 'weakening' your model. It ...
2
Email the authors.
Without some serious guesswork, there's no way I can think of to pull the unweighted data back out of a weighted data set like this without knowing the weights. For reference, here's what happened in that study:
Each patient has an exposure X, and outcome Y, and a set of confounding covariates Z
A model is built to estimate the ...
2
It seems surprising that the paper would not report the number of events, but if it reports the crude survival curves, either in a table or figure, then you can easily get the risk and thus the number of events, at any time t: risk(t) = 1-survival(t). Or if you have the overall survival probability: risk = 1-survival. Then the number of events is N*risk.
If ...
2
I will try to give an answer anyway (I looked at the image and you need the sample standard error of the mean):
The standard error is the standard deviation of the sampling
distribution of a statistic. Wikipedia
So
$$
SE_\bar{x}\ = \frac{s}{\sqrt{n}}
$$
and
$$
S = \sqrt{S^2} := \sqrt{\frac{1}{n-1} \sum_{i=1}^n{(X_i-\bar{X})^2}}
$$
So regarding the ...
2
Setting your stopping condition based on the significance of your interim analyses is, in general, not a great idea. The worst possible thing you could do would be to re-run your analysis after every page view and stop as soon as you got a significant result. You've decided, by setting your $\alpha=0.05$, that you're willing to tolerate a 5 percent chance of ...
2
As noted in my comment, I am a bit confused by what you are trying to do at the end but I am pretty confident that the answer is negative regardless. Several reasons for that:
You are neglecting errors/uncertainty in Marie's observations. You cannot directly conclude from a percentage of agreement (or a confidence interval on this percentage) that all the ...
2
Given the small number of categories (positive, neutral, negative) it will not take many to evaluate the relative performance of each algorithm. The specific number you'll need depends on the type of comparison you perform and the confidence and power you desire, but it's not worth over-thinking it at this point. I would start with 100 and you can always do ...
2
You have to formulate the hypothesis before you do any tests. The
hypothesis follows from your research question. In the case of the $t$-test, one possible two-sided hypothesis is the one you have given, i.e. that the difference between the two means equals some number. In most cases, $w=0$ and the $t$-test tests the null hypothesis that the two means are ...
2
I would use a Wilcoxon signed-rank test, separately for each question.
The paired t-test is appropriate if either 1) the data are distributed normally or 2) the sample size is large enough that the central limit theorem plausibly applies. I don't think either of these things is true here. 30 might be big enough for the CLT to apply or it might not.
I ...
2
I am not sure, but it sounds like your "summation" variable is quite similar to a proportion, in which case you can use a t-test; however, the "squared" aspect may mess that up; still the t-test is relatively robust.
I would first make plots; one thing to do here is parallel box plots. Another is to create two overlaid densities. If the variables look ...
1
I second @user21240's answer and @zbicyclist's comment but would like to offer a few additions:
First, the model @user21240 proposes models a linear course of the blood level over 11 hours time. We don't know what substance you are measuring, but if there could be diurnal cycles (e.g., for cortisol), you should account for these. My recommendation: ...
1
If you use Python, check out the Spearmint package on Github. The associated paper is available here. If you're familiar with Gaussian processes, it shouldn't be too hard to implement a basic version of the method yourself or to modify their code to suit your needs. The references in that paper may also point you in a good direction if it doesn't work in ...
1
Without knowing more about the data, especially the independent variables (the regressors), it will be impossible to give a one-fits-all solution for anyone here.
However the issue you are adressing falls into the area of Econometrics. As such, there are a couple of good texts which will bring you further.
They easiest book which will give you the tools to ...
1
You want to look into multinomial logistic regression. This is available in MATLAB via mnrfit. There is an outstanding discussion of MLR on CV here: Interpreting exp(B) in multinomial logistic regression.
1
This question reminded me of the FASTQC output....that is the result of scanning many short sequences (reads)....and looking for overrepresented motifs
From : http://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/11%20Overrepresented%20Kmers.html
This module counts the enrichment of every 5-mer within the sequence
...
1
In general the decision which significance level to take and when to reject depends on the prior knowledge of the researcher, which is a way to incorporate this knowledge in the test.
Read up on the p-value, which is basically what you are looking for. However, given that testing is highly dependent on the sample, the experiment or model, there are dangers ...
1
Since you say you have R I'll discuss it in terms of R. It can also be done in Excel, Matlab and according to pages I can see on the web, the regression tool in gnumeric will do multiple linear regression, so it should let you do this calculation.
The F-value is a test statistic, the p-value tells you about the probability of a test statistic at least that ...
1
This is a very interesting question, but one without an entirely satisfactory answer. The speculative content certainly increases as you go down.
One problem with randomized experiments is that they are not valid in the presence of what economists call "general equilibrium effects," which are interactions among individuals induced by the treatment being ...
1
What's the implicit model?
The probability p = 1/1024 is derived from the fair coin model of $\Pr(H)=\Pr(T)=0.5$ with independent throws, i.e. $Cov(n_i,n_{i-1})=1$. Note that this model is invariant to the sequence of throws, i.e. $\Pr(HHT)=\Pr(HTH)=\Pr(THH)$.
Under this model, a sequence such as HTHTHTHT is suspect, because it violates the second ...
1
This is a useful text on "Abrupt change detection" by Michèle Basseville and Igor V. Nikiforov,. I think it can answer the substance of your question. I think the CUSUM methods might do well there.
EDIT:
I am not sure about the method you describe. After some thought, it sounds like you are asking about a correspondence between the two. If so then ...
1
First you need to decide whether to treat the responses as if they are at the interval or ordinal level of measurement. "Interval" would assume that the difference between 1 and 2 is the same as the difference between 2 and 3 (and so on all the way to the difference between 9 and 10). "Ordinal" would relax that assumption.
With data treated as interval, ...
Only top voted, non community-wiki answers of a minimum length are eligible
