2
$\begingroup$

I am new to statistics and Bayesian analysis. Therefore, I have some problems that would like to clarify.

Suppose my problem is to calculate the posterior distribution for the time of ship to spend in port. Because there is no historical distribution regarding how long a ship stays in that port, therefore I assumed a uniform distribution [1, 15], with 1 hour denotes the lower bound, and 15 hours denotes the upper bound. This range includes the basic ship operation and cargo handling in port e.g., a ship could arrive and finish cargo handling within 15 hours. Then, I collected some actual ship arrival samples. I collected 50 ship samples that represent the current situation of port time for a ship. Based on the samples, I understand they follow a normal distribution [15, 5] with mean 15 hours, and standard deviation 5 hours.

Base on the information above, I would like to confirm if my understanding on Bayesian is correct:

  1. Think of the time as observations coming from some distribution, for example, Gaussian with some parameters mu and sigma. If mu and sigma are known, then we can always sample observations from that distribution, and there is no need to perform statistical analysis. However, the values of mu and sigma are usually unknown, and we perform statistical analysis to estimate them. In the frequentist approach, in which parameters mu and sigma are just points, we can approximate them using the sample mean and sample variance. In the Bayesian paradigm, things are a little bit different. Here we assume that mu and sigma have prior distribution on their own. So one of the goals of Bayesian is to estimate the posterior distribution of mu and sigma given the evidence, which is sample data in this case. Generally, we do not estimate the posterior distribution of data or time in the example. We estimate the posterior of the parameters of the distribution that generates this data.
  2. If my above understanding is correct, I am not sure how to set the prior for the mu and sigma. Given I have the prior uniform distribution [1, 15], can I set the prior distribution of mu to follow the uniform distribution [1, 15], then set a Jeffrey prior for the variance? Or, from your opinion, what prior should I use for variance?

Thank you very much for your kind help. I am really lost about the next step.

$\endgroup$

1 Answer 1

3
$\begingroup$

Welcome to the Bayesian world! I think your confusion comes from not separating the likelihood and priors. When you develop a Bayesian model, it is always a good practice to develop it line by line. By this, I mean like:

\begin{equation} \begin{split} y_{i} &\sim N(\mu, \sigma),\\ \mu &\sim U(1, 15),\\ \sigma &\sim Exp(1), \end{split} \end{equation}

where the first line is likelihood, i.e., the distribution of the data you have, while the second and third lines are priors, i.e., the prior distribution of parameters, $\mu$ and $\sigma$ here. The above model means:

"I think data come from the normal distribution. I don't know what the mean of this distribution is, but from my intuition, it should be any value between 1 and 15 with all values being equally probable. I don't know what the standard deviation of the normal distribution is either, but from my intuition, it should be any positive value but different probability following the exponential distribution with the rate = 1."

Generally, priors express your uncertainty of the true parameter value (if such a thing exists). There is no hard-coded rule on how to specify a prior, but there are several versions such as noninformative priors, weakly informative priors, and informative priors, and the specification all depends on what you know about the parameters of interest beforehand. Sometimes, you can reason why certain values are impossible (e.g., the waiting hours cannot be a negative value, right?). Then, the data updates your priors to posteriors, so you obtain the posterior distribution of $\mu$ and $\sigma$, $p(\mu | D)$ and $p(\sigma | D)$.

Then, if you want to calculate "the posterior distribution for the time of ship," you draw $y_{i}$ from $N(\mu, \sigma)$, where $\mu$ is drawn from $p(\mu|D)$ and $\sigma$ from $p(\sigma|D)$ (the two posteriors could be correlated, in which case, you would draw values from the joint distribution). The results are the "posterior predictive distribution": $p(\hat{y}|\mu,\sigma)$. It is usually wider than the distribution of predicted values in the Frequentist world, because there $\mu$ and $\sigma$ are fixed values.

$\endgroup$
5
  • $\begingroup$ Hi Patrick, thank you very much for your answer. May I just confirm that my understanding is correct: in Bayesian analysis, we are not trying to calculate the posterior distribution of the data. Instead, we calculate the posterior distribution for each model parameter. To achieve that, I have some prior knowledge on mu and sigma separately, then I use this prior mu and prior sigma to update the likelihood mu and sigma (e.g., collected sample). In the end, I get the posterior mu and sigma, which could be used to draw samples of the updated port time. Am I correct? $\endgroup$
    – user7
    Dec 9, 2021 at 0:02
  • $\begingroup$ In my case, I do not have any prior knowledge on how long a ship used to stay in a port, therefore, I used a uniform distribution [1, 15] to assume that a ship stays for minimum 1 hour and maximum 15 hours to finish cargo loading/unloading. In this case, I used a uniform prior for mu, and Jeffreys prior for sigma. However, I am not so sure that my prior choices are feasible. From your opinion, what prior do you think is suitable for my case? Basically, I wish that the final posterior distribution could be mainly based on the sampled data (likelihood function) :) $\endgroup$
    – user7
    Dec 9, 2021 at 0:05
  • $\begingroup$ I am not entirely sure what you mean by "the posterior distribution of the data." The distribution of data is what you have, and will be updated as you collect more data, which is outside statistical modeling. You can compute the posterior "predictive" distribution of data, which is analogous to computing fitted values in Frequentist statistics (but with more uncertainty usually in Bayesian statistics as I already wrote). $\endgroup$
    – Patrick
    Dec 10, 2021 at 14:29
  • $\begingroup$ "update the likelihood mu and sigma" should be "update the mu and sigma in the likelihood." "draw samples of the updated port time" should be "draw samples of the posterior predictive distribution," which is NOT updated data (again, you can update your data only by collecting more data). $\endgroup$
    – Patrick
    Dec 10, 2021 at 14:30
  • $\begingroup$ While you say you "do not have any prior knowledge," you know (and I know) the waiting time must be >= 0, right? Otherwise, you would put a prior of U(-inf, inf) for mu, which would be absurd (and I realized that the normal dist. is not a right likelihood as it is not constrained between 0 and positive values; it should be log-normal or gamma or whatever that takes only values >=0). Finally, if you want data to dominate the posterior, use noninformative priors (but make sure some apparently noninformative priors become informative after parameter transformation). $\endgroup$
    – Patrick
    Dec 10, 2021 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.