I was wondering how to calculate the following joint distribution by assuming that $x_i$'s are continuous observations from normal distirbution $N(\mu, \sigma)$ with mean $\mu$ and variance $\sigma$ and $p(\mu, \sigma)$ is the joint distribution of $\mu$ and $\sigma$: $$ p(x_1,x_2,..., x_n, x_{n+1}) = \iint {\displaystyle \prod_{j=1}^{n+1} N(\mu,\sigma)|_{x_j}} . p(\mu, \sigma) \,d\mu \, d\sigma $$ I know that this can be accomplished by solving the intergration numerically (e.g. with rectangular integration), however, the thing that puzzles me is that the products of $N(\mu,\sigma)|_{x_j}$ would be always zero since the probability of the point $x_i$ in a continuous distribution $N(\mu,\sigma)$ is zero (or I'm wrong?).

In addition, what is the best method to calculate (or estimate) this distribution in python?


Ok, this looks like deep learning related stuff, maybe an exercise.

$N(\mu,\sigma)|_{x_j}$ is not the probability of a point $x_j$, this is the density of point $x_j$ given the parameters $\mu, \sigma$. The total integral is not the probability either, it is the likelihood of sorts or a density function. The whole thing is in fact never a zero, because the density $N$ of normal distribution is never zero, it can be very small of course.

  • $\begingroup$ Thanks. Do you have any suggestion for the implementation in python? I've used dblquad for the integration but it takes a lot of time for evaluating a series of let's say 20 observations. I think the slowness of the integration is related to the fact that the p(μ,σ) is calculated using the product of two log-normal distribution but I simply use linear (not exponential) interval for the integrands. $\endgroup$ – farzad Feb 18 '18 at 16:40
  • $\begingroup$ I'm not sure someone intended to integrate this numerically. They may have written the equation to get across the concept. Multidimensional integration is very difficult beyond 2-3 dimensions, it's heavy computationally. You may want to integrate analytically or re-consider whether you need to integrate at all. If need help on integration though, it's better to post a separate question. There's a chance the question would be considered off-topic and moved to other SE or SO sites $\endgroup$ – Aksakal Feb 18 '18 at 17:37
  • $\begingroup$ But this formula comes from a paper in which the authors used numerical integration via a simple quadrature rule. Since the integrands are represented by log-normal distributions, they have used a rectangle rule in which the intervals between interpolation points vary exponentially. But I don't know how to use the rectangular rule of integration with exponential intervals for the interpolations. $\endgroup$ – farzad Feb 18 '18 at 17:54
  • $\begingroup$ What was dimensionality? $\endgroup$ – Aksakal Feb 18 '18 at 17:55
  • $\begingroup$ The observations have only one dimension. $\endgroup$ – farzad Feb 18 '18 at 18:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.