Is the t-test to test the difference from 0?

Question

I have a variable x with a mean 1 and sd of 3. So most of the mass is above zero. Let's say I have n=100 observations. A t-test will tell me whether the mean of x E(x) is different from 0 or above 0. It will give me a confidence interval, say in [.8, 1.2] in which I can find the mean.

The higher n is, the more confident I become in this.

But what I really want to know is whether (the distribution of) x is mostly above 0. In my case, about 60% of x is positive.

What would be a good test, Bayesian or frequentist, to measure this?

Answer 1

I will leave the frequentist methodology for someone else to write about. I will give you a Bayesian technique. From a Bayesian point of view, you're interested in the posterior predictive distribution. Mathematically,

$$ Pr(\tilde{y} \vert y) = \int _\theta Pr(\tilde{y} \vert \theta)Pr(\theta \vert y) \, d\theta \>. $$

Here, $\tilde{y}$ is "data that we would have seen tomorrow if the experiment that generated y today were replicated with the same model and the same value of $\theta$ that produced the observed data" (Gelman, BDA 3). If we could get the distribution of $\tilde{y}$, we could estimate the probability that the data are greater than 0.

If you're willing to make strong assumptions on the likelihood of the data, and have prior information on the parameters for the likelihood, a Bayesian technique is more or less straightforward. You could do this pen an paper, assuming you use conjugate priors. For example, assuming a normal likelihood, a normal prior for the mean, and an scaled inverse chi squared for the variance, its straight forward to show the posterior predictive distribution is Student t. Else, you could use Stan or other Bayesian software.

Bayesian softeware does not give you a distribution but rather samples from the posterior (I'm being somewhat fast and loose here, but its fine for your purposes). Probability calculations come in the form of integrals, but if we have samples from that distribution, all we have to do is count the proportion of samples which are above 0.

Here is an example in R using rstanarm (a library which wraps some Stan code to do common statistical models like GLMs). Assume I have some data which I suspect comes from a normal distribution. I can use a guassian GLM to fit both the mean and the variance. Then, I can generate data from the posterior predictive, and estimate the probability that a new sample is above 0.

library(rstanarm)

# Generate data
set.seed(4)
mu = rnorm(1)
sigma = rgamma(1, 2, 2)
x = rnorm(100, mu, sigma)

# Here is the data you might start with
d = data.frame(x=x)

# Fir a gaussian GLM
# Since I have no predictors, this will fit the mean and variance
model = stan_glm(x~1, family = gaussian(), data=d)

# Generate from the posterior predictive distribution
ytilde = posterior_predict(model, newdata = data.frame(x=1))

# We can plot a histogram of the predicted density
hist(ytilde)

# Compute the probability we see something above 0
mean(ytilde>0)
>>>0.69...
```