How to set prior for Bayesian analysis

Question

I am new to statistics and Bayesian analysis. Therefore, I have some problems that would like to clarify.

Suppose my problem is to calculate the posterior distribution for the time of ship to spend in port. Because there is no historical distribution regarding how long a ship stays in that port, therefore I assumed a uniform distribution [1, 15], with 1 hour denotes the lower bound, and 15 hours denotes the upper bound. This range includes the basic ship operation and cargo handling in port e.g., a ship could arrive and finish cargo handling within 15 hours. Then, I collected some actual ship arrival samples. I collected 50 ship samples that represent the current situation of port time for a ship. Based on the samples, I understand they follow a normal distribution [15, 5] with mean 15 hours, and standard deviation 5 hours.

Base on the information above, I would like to confirm if my understanding on Bayesian is correct:

1. Think of the time as observations coming from some distribution, for example, Gaussian with some parameters mu and sigma. If mu and sigma are known, then we can always sample observations from that distribution, and there is no need to perform statistical analysis. However, the values of mu and sigma are usually unknown, and we perform statistical analysis to estimate them. In the frequentist approach, in which parameters mu and sigma are just points, we can approximate them using the sample mean and sample variance. In the Bayesian paradigm, things are a little bit different. Here we assume that mu and sigma have prior distribution on their own. So one of the goals of Bayesian is to estimate the posterior distribution of mu and sigma given the evidence, which is sample data in this case. Generally, we do not estimate the posterior distribution of data or time in the example. We estimate the posterior of the parameters of the distribution that generates this data.
2. If my above understanding is correct, I am not sure how to set the prior for the mu and sigma. Given I have the prior uniform distribution [1, 15], can I set the prior distribution of mu to follow the uniform distribution [1, 15], then set a Jeffrey prior for the variance? Or, from your opinion, what prior should I use for variance?

Thank you very much for your kind help. I am really lost about the next step.

Answer 1

Welcome to the Bayesian world! I think your confusion comes from not separating the likelihood and priors. When you develop a Bayesian model, it is always a good practice to develop it line by line. By this, I mean like:

\begin{equation} \begin{split} y_{i} &\sim N(\mu, \sigma),\\ \mu &\sim U(1, 15),\\ \sigma &\sim Exp(1), \end{split} \end{equation}

where the first line is likelihood, i.e., the distribution of the data you have, while the second and third lines are priors, i.e., the prior distribution of parameters, $\mu$ and $\sigma$ here. The above model means:

"I think data come from the normal distribution. I don't know what the mean of this distribution is, but from my intuition, it should be any value between 1 and 15 with all values being equally probable. I don't know what the standard deviation of the normal distribution is either, but from my intuition, it should be any positive value but different probability following the exponential distribution with the rate = 1."

Generally, priors express your uncertainty of the true parameter value (if such a thing exists). There is no hard-coded rule on how to specify a prior, but there are several versions such as noninformative priors, weakly informative priors, and informative priors, and the specification all depends on what you know about the parameters of interest beforehand. Sometimes, you can reason why certain values are impossible (e.g., the waiting hours cannot be a negative value, right?). Then, the data updates your priors to posteriors, so you obtain the posterior distribution of $\mu$ and $\sigma$, $p(\mu | D)$ and $p(\sigma | D)$.

Then, if you want to calculate "the posterior distribution for the time of ship," you draw $y_{i}$ from $N(\mu, \sigma)$, where $\mu$ is drawn from $p(\mu|D)$ and $\sigma$ from $p(\sigma|D)$ (the two posteriors could be correlated, in which case, you would draw values from the joint distribution). The results are the "posterior predictive distribution": $p(\hat{y}|\mu,\sigma)$. It is usually wider than the distribution of predicted values in the Frequentist world, because there $\mu$ and $\sigma$ are fixed values.