I'm currently trying to implement a Mixture Density Network (MDN) based off of the original paper here.

Most of the equations seem pretty straight forward but on page 6 (7 of the PDF) equation 23 has a variable 'c' that isn't explained. The only other place I found a mention to a 'c' is at the end of page 7 (8 of the PDF) where it is used as the number of outputs from the network if you weren't using an MDN, but it seems like it would be strange to use that if your outputs are independent.

I found a python implementation here. It seems like the line:

    ps = np.exp(-((y - mu)**2)/(2*sig**2))/(sig*np.sqrt(2*math.pi))

is where Equation 23 is used and 'c' looks to be 1 in this case. Is that because this is a single mapping from x to y so the original network only had one output?

That one line is also a little weird because according to equation 23 shouldn't the (y - mu)**2 portion actually be the dot product of (y - mu) with itself? I'm definitely confused by the mu vector, is that the vector of kernel means used in that mixture?

Say you have 3 outputs from your non-MDN network (x, y, z). My understanding is you could set up an MDN in different ways, for example:

A mixture of M tri-variate kernels

M * 3 uni-variate kernels, mixture of M kernels for each output

In both these cases would you have the same number of MDN parameters? (M * 3 means, M * 3 variances, M * 3 mixture coefficients)

  • $\begingroup$ (y-mu)**2 literally means (y-mu)*(y-mu) $\endgroup$ – Glen_b Apr 29 '16 at 3:35
  • $\begingroup$ I don't have any experience in python so I wasn't sure if that was an element-wise multiplication or a dot product. $\endgroup$ – Dukes_Man Apr 29 '16 at 4:44

$c$ denotes the dimension of an output variable. See the first sentence of the first section, on the first page.

  • $\begingroup$ Thanks for that. I was pretty sure that was the case, but somehow I missed that first mention of it. $\endgroup$ – Dukes_Man Apr 29 '16 at 4:45
  • $\begingroup$ no sweat. cool looking paper though, I think I'm going to take a look at that later myself $\endgroup$ – Taylor Apr 29 '16 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.