I am reading research done in this thesis. On page 20-21 (section 2.1) of the thesis, it describes the following stochastic model:

... let us focus for the rest of this chapter on time-varying signals ... In this context, an observed object is simply a signal in a time window of a certain length. A latent source object may be thought of as a signal corresponding to a prototypical type of event. If the same type of event were to happen many times, we suppose that the resulting observed signals are noisy versions of the latent source signal corresponding to that type of event.

We say an observation $s$ from an infinite stream $s_{\infty}$, is generated by a latent source $q$ if $s$ is a noisy version of $q$. Accordingly, we propose the following stochastic model relating a latent source $q$ and an observation $s$: $$P(s \text{ generated by } q) \propto \text{exp}(-\gamma d(s,q))$$

where $d(s, q)$ is the distance between $s$ and $q$ and $-\gamma$ is a scaling parameter.

I had never seen this formula before, and after looking around, I think it may be related to a potential function, whose representation is the Boltzmann distribution. There is a picture on page 23 of the thesis depicting what seems like different potential values.

Am I correct? Any other insights/references are welcome.


The Boltzmann distribution is not only a different expression (it shares the exponential term, but so do many other distributions), but also a much different mechanism that generates the distribution (The Boltzmann distribution is more like related to a combinatorics problem relating probability of an event, observing some energy or energy distribution, to the size/density of the set of points in the sample space with that energy distribution).

The distribution that you have, when using the squared distance $d (s,q) = \sum_{i=1}^{N_{obs}} (s_i-q_i)^2 $ looks more like a multivariate Gaussian distribution.

When you substitute that term then the equation becomes: $$P(s \text{ generated by } q) \propto \prod_{i=1}^{N_{obs}} \text{exp}(-\gamma (s_i-q_i)^2)$$

I haven't read much more from that thesis but it looks like some sort of generalization of k-means clustering or mixture model.

  • 1
    $\begingroup$ this looks like some kind of a kernel method $\endgroup$ – Aksakal Dec 30 '18 at 15:42
  • $\begingroup$ I am not sure how you connect the euclidean distance formula with the multivariate Gaussian distribution. Also, I looked into some online k-means clustering and I didn't see any references that seemed similar, can you provide one that does? $\endgroup$ – guy Dec 30 '18 at 19:33
  • $\begingroup$ @guy A multivariate distribution can be related to the Euclidean distance by considering a geometrical interpretation en.m.wikipedia.org/wiki/… $\endgroup$ – Sextus Empiricus Dec 30 '18 at 20:18
  • $\begingroup$ @MartijnWeterings I did some further looking into and I've come to the conclusion that this is a spin on the radial basis function. Since this is related to SVM kernels, k-means clustering, and other critical topics, it is confusing that the authors never mentioned this. $\endgroup$ – guy Jan 10 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.