New answers tagged statistical-significance
0
Assuming that all people make independent choices, we can model the experiment as draws $X_i$ from a Bernoulli($p$) distribution with $E(X_i) = p$ and $\sigma^2(X_i) = p(1-p)$. For large $n$ ($n = 1000$ in your case) the independent draws from the Bernoulli distribution can be modelled as a Normal distribution according to the Central Limit Theorem. We then ...
1
Since you are uncertain about what may be the true conversion rate of the ad, you should consider using bayesian approach:
Let $X$ be an r.v such that $X=1$ if the ad is successful. So $X \sim Be(p)$, where $p$ is the conversion rate of the ad and let's assume further that $p \sim U(0,1)$. So basically you have a no prior belief about how efficient the ad ...
1
The classical parametric approach in frequentist statistics would be to model the situation as follows. Suppose all the people who see a certain advertisement are all independent of each other and have the same probability $p$ of clicking on the advertisement. We say these people form a Bernoulli population, i.e. a member $X$ of this population either reacts ...
2
Confidence intervals are formed by the "inversion" of probability statements about pivotal quantities. There are a few ways you can get a confidence interval that encompasses impossible values for the parameter it is estimating. One common way this occurs is when you use an approximation to the true distribution of the pivotal quantity (e.g., an ...
3
Some of the confidence intervals for binomial success probability $p$ are derived
from asymptotic expressions and they should not be expected to work well for small $n.$
As an example, suppose I observe $n=15$ binomial trials and find $X = 1$ Success.
The Wald CI, based on an asymptotic argument, uses point estimate $\hat p = x/n = 1/15.$
Then the 95% Wald ...
1
No, you can not use the z distribution when you have a small sample size like that. The distribution of sample means approaches the normal distribution as the sample size gets larger. With small samples, the distribution isn't very close to the normal distribution, so approximations using z-scores won't be very accurate.
The normal distribution is used to ...
1
Yes, you are likely (extremely likely) to get slightly different results each time, just due to random sampling. It sounds like you’re doing it right. Just use a seed (e.g. set.seed in R or np.random.seed in Python) in your work so others can reproduce your results. That makes it so you get the same random sample each time you run your code.
0
It is not clear to me if a length of a meow of a cat is just an example or the real context, but will treat it as the last. You cannot just say that you are not interested in the variation between cats (even if that is not the inferential interest), since that contributes to the variability, creates dependence in the data. So just ignoring it could ...
0
The Root Mean Squared Error is the square root of summed squared errors. It is in a way an "expected error". You could say "on average, the error between the observed data and the model prediction will be 5 units". Could it be smaller/larger? Yes. But on average it would be 5.
1
Thresholds based on mean and standard deviation are reasonable if you know that your data is approximately normally distributed. While there are hypothesis tests for normality, these might not yield useful results for outlier detection, because you are interested in the variable range with very low probabilities, i.e. the long tails of the distribution. If ...
1
Treat each joint distribution as a simple categorical one with 9 levels, this way you can compare them with distances like Hellinger, Jensen-Shannon etc. and perform Chi square or G test.
This is clearly the best option for your case, if you want to consider the joint distribution in particular, without giving any weight to affine combinations of categories, ...
1
If you have the data to place the orders in a contingency table then you can compute conditional probabilities of selecting the other items. Based on that you can make decision rules to add the items to the list.
For instance a hypothetical table, showing the frequency of items being bought together (where X means that the item is bought without another item)...
5
Your question is circular. You say you want to use a test find the 'important' values. And when asked what you mean by 'important' you say the 'important' values are ones that pass the test.
To make sense of this it might be most useful for you to look at numerical or graphical summaries of your data.
Then from the summaries try to get an intuitive view of ...
0
You may fit a loglinear model:
https://en.wikipedia.org/wiki/Log-linear_analysis
0
Indeed, the fractions of suicide $\frac{3}{83}$ and $\frac{0}{84}$ are different. That is correct. But for a good research report, we also want to express how much this observation may have been due to chance.
There are all sorts of ways to compare these numbers. To get these differences intuitively. Maybe it is more intuitive/straightforward to do some ...
0
You should describe more precisely what you mean by this,
I now care about what essentially amounts to the conversion rate from each $ spent to generate the conversions.
You are maybe describing a model where money spent influences the conversion rate? Or you just know exactly which model costs what and want to make a decision? But are the costs total and ...
1
Several frequently used styles of confidence intervals, especially CIs that are intended for use with large samples, happen to be based on the central limit theorem. But many confidence intervals have no particular relationship to the CLT.
CIs for binomial proportion. The Wald CI for the Success probability $p$ of a binomial model is based on the normal ...
0
I realize I have a data setup issue.
2
You can perform the goodness of fit test. Given two vectors of data you test, through the chi-squared test, if they are significantly different or, given a vector of data, you test if their frequencies significantly differ from a given vector of probabilities.
Data comparison:
x1 = c(100, 300, 400, 200)
x2 = c(101, 302, 399, 202)
chisq.test(x=x1,y=x2)
...
0
No, you’re not guaranteed to nail the population value, not with bootstrap or with any other estimation method. All of the usual caveats about estimation (bias, variance...) apply to procedures based on bootstrap.
If you had some bad luck and drew a misleading sample from your population, you’re out of luck when it comes to estimation, but that’s always true....
0
Generally speaking, the false discovery rate correction requires ranking your p-values and then computing a critical value (based on the false discovery rate) that you can compare to your p-values. This website gives a very concrete example: here.
If you're using R, then it's even easier to do the adjustment. The following code is a quick example:
pvals <-...
0
I was able to resolve the problem with p-values > 1 by going to the original R code (https://github.com/amices/mice/blob/master/R/mipo.R#L71) which shows that the linked post missed the abs. The final test shoud look like this:
pt(q = abs(pooledMean / pooledSE), df = nu_BR, lower.tail = FALSE) * 2
After talking to some statisticians, we've come to the ...
1
It sounds like you just want a correlation matrix.
For x columns, this measures the correlation between each column's data.
Here, (Pearson's) correlation is a normalised version of the covariance of any two variables, so you don't need to worry about units.
In R, just read in your data frame with the 6 score columns. Since you want to check for significant ...
1
The question is seeking for the probability
$$p=P(Z>1.98\cup Z<-1.98)=P(Z>1.98) + P(Z<-1.98)$$
And, the Z-table linked is most probably the CDF of standard normal RV, i.e. $F_Z(z)=P(Z\leq z)$. Since this is symmetric with respect to origin, you have $P(Z>1.98) = P(Z\leq -1.98)$. Since the mentioned z-table gives us the probabilities for any $\...
2
There are many ways to combine $p$-values, in fact we even have a tag for them combining-p-values. One method which is often used is Fisher's method which does, in effect, multiply the $p$-values. In fact logs are taken and summed which comes to the same thing. The crucial difference though is that this does not yield the new $p$-value directly but rather it ...
2
You can, I suppose, use a product of $p$-values $P \equiv \prod p_i$ as a measure of evidence if you like.
But it isn't itself a $p$-value, as for one thing it isn't uniformly distributed between 0 and 1 under the null, i.e., we don't have $P \sim U(0, 1)$ under the null. So it cannot be used to control type-1 errors in the usual manner. If we reject when $...
1
Could you use the multiplication rule from probability to multiply the
first p-value by the second, thus getting a new overall probability of
seeing your test statistic?
e.g. you do a t-test and get a p-value of 0.05, and then you perform
the same test with a completely different sample and get a p-value of
0.10. ...
[p-value= $ 0,05 * 0,1 = 0,005$?]
No, it ...
11
"I believe the interpretation of the p-value is that it is the probability of seeing your sample's test statistic under the null hypothesis."
No. It is the probability to see your sample's test statistic or something that is even less in line with the null hypothesis ($H_0$) under the $H_0$, which I write as $P_0\{T\ge t\}$, where $T$ is the test ...
2
Let $X \sim N(\mu,\sigma^2)$ and $\{x_i\}$ and $\{y_i\}$ be two same size i.i.d samples from $X$.
Now we do z-tests (assuming variance is known) individually for the two samples and then together. In both cases the null hypothesis is same:
$$H_0: \mu=0$$
Let $p_x$ and $p_y$ be respective p-values for individual tests and $p_{xy}$ be the p-value for combined ...
7
No, because then you could repeat any insignificant p-value and get a significant result, e.g. $0.9^{100}\approx 0.0000027$. Fisher's method is one way to combine multiple p-values.
1
This is the problem of multicollinearity. When one variable lies close to the space spanned by the others then the p value will be split among them diluating its value on any one. The variance inflation factor (VIF) can be used to diagnose this. Typically a value above 5 is regarded as problematic and that is the case here.
library(faraway)
vif(mod1)
## ...
1
It is a known issue in regression with correlated predictors that the standard errors on the coefficients get inflated, resulting in larger p-value and decreased power to reject a null hypothesis that a particular parameter is zero. You still can wind up with the Gauss-Markov theorem in effect to give you that unbiased estimator, but inferential ideas (p-...
0
In most types of regression, typical approach to compare the effect of a program (drug type) on certain outcome is to make one model for all the observations, and a dummy (0-1) variable for treatment:
$cells_i = \beta_{0} + \beta_{1}drug\_type_i + \varepsilon_i$
If needed, other control variables may be included, such as:
$cells_i = \beta_{0} + \beta_{1}drug\...
1
Here are the probabilities for $0, 1, \dots, 10$ heads if we throw a coin $n=10$ times under your null hypothesis of $p=0.3$:
So let us assume that we have observed $n=5$ heads and wish to run a two-sided test. I have indicated the probability for observing $k=5$ under the null hypothesis of $p=0.3$ with the horizontal red dashed line. Take a look at the ...
1
I do not know your particular numbers, but you may have encountered an example of Simpson's paradoxon. Arithmetically, it boils down to the observation that
$$\frac{s_{11}}{n_{11}} < \frac{s_{21}}{n_{21}} \quad\mbox{and}\quad \frac{s_{12}}{n_{12}} < \frac{s_{22}}{n_{22}}$$
does generally not imply
$$\frac{s_{11} + s_{12}}{n_{11}+n_{12}} < \frac{s_{...
2
This is a standard power analysis. To determine how many samples you need in each setting, you need to specify: a baseline value of interest, the difference between values that you want to detect, the chance that you are willing to accept a false-positive difference (Type-I error), and the chance that you are willing to accept that you miss a true-positive ...
0
Logistic regression is suitable to the analysis of binary data, wherein you can have binary independent as well as dependent variables. SPSS, and R are good options to do this. Chi square might also be an option here.
1
The distribution of the number of 1's in each group is a binomial distribution, since it's a count of iid failures/successes. You can find information about the adequate statistical rest here. You can easily simulate this process: just think about the number of samples from each group and the probabilities of getting a 1 from each group and use these ...
0
You could use chi square as you propose, or maybe a binomial glm with time as covariate.
For a similar example, see Testing for significant difference in mortality rate between multiple groups
1
If your data are a random sample from a population (as opposed to the population itself), then it seems a one-sample Wilcoxon test would
show that the center of the population sampled differs from the center of
your simulation (average value line).
In particular actual settlements seem to prefer 0-200 m, whereas your simulation shows many locations around ...
2
Even stipulating that the causal interpretation ('increase') is valid, it's entirely possible that $B$ both increases and decreases $y$.
The original $t$-test compares arithmetic means; the log model compares geometric means. There is nothing especially weird about a binary variable increasing one and decreasing the other.
The mean is more affected by the ...
0
There is actually widespread confusion about how to interpret p-values, even among scientists. I recommend the ASA Statement on Statistical Significance and P-Values and Greenland et al. (2016) as references.
Welch Two Sample t-test
data: lizard$cold and lizard$warm
t = -1.7796, df = 10.147, p-value = 0.1051
alternative hypothesis: true difference in ...
0
This looks like a homework problem. I assume the course is over now. The contrast asked about is #3, with coefficients 1 for the bright group and -.5 for each of the other two groups. The difference between the mean for the bright group and the average of the other two is in the Value of Contrast column. The p value or significance associated with it is .001 ...
0
A very simple example of a random variable $x$ and a function $f$ such that $Cov(x, f(x)) = 0$ would be $x$ and $x^2$ for any $x$ with a symmetric distribution (and finite second moment), as suggested by @PedroSebe below. In such cases, $Cov(x, f(x)) = 0$ because $E[x|x^2] = 0$.
Alternatively, what follows is a less ad hoc formulation of the example ...
1
Generate $n$ samples of $X \sim \mathcal N(0,1)$ and $Y \sim \mathcal N(0,1)$.
Define $Y'$ as the residuals of the least squares linear model $Y=\beta_0+\beta_XX+\epsilon$.
Residuals are guaranteed to be uncorrelated with the independent variable.
Or, in other words, $Y'=\epsilon= Y - \beta_0 -\beta_XX$.
Since $\epsilon \sim \mathcal N(0, \sigma^2)$, you ...
0
Increasing the number of civil wars would not necessarily change the significance. For instance, if the number of civil wars doubled for each country, your significance level should remain the same (this is also something you can test yourself with your data!).
Below is a good recourse. It covers both the logit and probit models, which are both very similar.
...
1
You can use $y=x^2$. The symmetry of this function guarantees that the correlation will be zero. Keep in mind that the correlation might not be zero in any particular sample you generate, but it will be close to zero as long as your sample size is large enough.
0
Let $y$ be the fractional part of $x \cdot 1000000$. For any reasonably smooth $x$, this is nearly uncorrelated with $x$.
6
Despite the title -- which is software-specific -- there are some statistical issues within this thread.
A probability density necessarily integrates to $1$ over the range of the data. In your case the empirical range is about $1$ or $2 \times 10^{-3}$ or more plainly about $0.001$ or $0.002$. So the average density should be between $1000$ and $500$, which ...
3
You are possibly mixing up significance and effect size. While the difference between these two distributions is small, the fact that you have large samples makes even such a small difference very unlikely to happen by chance, and therefore significant. This is all but unusual.
Top 50 recent answers are included
