1
$\begingroup$

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, Wood-bury matrix inversion lemma is used. \begin{equation} \mathcal{L} = \sum_{n=1}^{N} \log \sum_{i=1}^{K} \pi_i P(x_n | \mu_i, \Sigma_i) = \sum_{n=1}^{N} \log \sum_{i=1}^{K} e^{[log (\pi_i) + \log P(x_n | \mu_i, \Sigma_i)]} , \end{equation} where, \begin{equation} log P(x | \mu, \Sigma) = -\frac{1}{2}\big[d \log(2\pi) + \log \det(\Sigma) + (x-\mu)^{T} \Sigma^{-1} (x-\mu)\big] \end{equation} \begin{equation} \Sigma^{-1} = D^{-1} - D^{-1} A L_{l \times l}^{-1} A^T D^{-1} \end{equation} \begin{equation} \log \det(AA^T+D) = \log \det(I + A^T D^{-1} A) + \log \det D = \log \det L_{l \times l} + \sum_{j=1}^{d} \log d_j \end{equation} I do the following:

K_exp_sum = 0
logK_exp_sum = 0
for i in xrange(k):
    dvar_slice= tf.slice(dvar, [i, 0], [1, num_features])
    D         = tf.squeeze(tf.diag(dvar_slice), axis=[0, 2])
    Din       = tf.squeeze(tf.diag(1/dvar_slice), axis=[0, 2])
    Asl       = tf.squeeze(tf.slice(A, [i, 0, 0], [1, num_features, latent_dim]), axis=0)
    ATDin     = tf.matmul(tf.transpose(Asl), Din)
    L         = tf.eye(latent_dim)+ tf.matmul(ATDin, Asl)
    Sigin     = Din - tf.matmul(tf.matmul(Din, Asl), tf.matmul(tf.matrix_inverse(L), ATDin))
    logdetL   = tf.log(tf.clip_by_value(tf.matrix_determinant(L), 1e-10, np.inf))
    logdetD   = tf.log(tf.clip_by_value(tf.reduce_prod(dvar_slice), 1e-10, np.inf))
    const     = num_features*tf.log(2*3.1415926535)
    logphi    = tf.transpose(tf.log(tf.slice( tf.matmul(phi, tf.ones([1, batch_size])), [i, 0], [1, batch_size])))
    xSiginxt  = tf.diag_part(tf.matmul(tf.matmul(X-K_centers[i], Sigin), tf.transpose(X-K_centers[i])))
    K_exp     = tf.exp(-0.5*(tf.reshape(xSiginxt, [batch_size, 1])+logdetL+logdetD+const)+logphi)
    K_exp_sum = K_exp_sum+K_exp
    # print("Step %i/%i completed"% (i, k))
logK_exp_sum  = logK_exp_sum+tf.reduce_sum(tf.log(tf.clip_by_value(K_exp_sum,1e-10,np.inf)))
logK_exp_sum = -logK_exp_sum
optimizer = tf.train.AdamOptimizer(LEARNING_RATE).minimize(logK_exp_sum, global_step=global_step)

I get the following error:

InvalidArgumentError: The determinant is not finite.
    [[Node: Model_likelihood/MatrixDeterminant = MatrixDeterminant[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Model_likelihood/add/_81)]]
    [[Node: Model_likelihood/MatrixDeterminant_3/_71 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_1280_Model_likelihood/MatrixDeterminant_3", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

Caused by op u'Model_likelihood/MatrixDeterminant', defined at:
File "/home/user/bin/anaconda2/envs/condaenv/bin/ipython", line 11, in <module>
    sys.exit(start_ipython())
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/__init__.py", line 119, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 355, in start
    self.shell.mainloop()
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 495, in mainloop
    self.interact()
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 486, in interact
    self.run_cell(code, store_history=True)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2714, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2818, in run_ast_nodes
    if self.run_code(code, result):
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2878, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-1-da7fdb6d9d66>", line 65, in <module>
    logdetL   = tf.log(tf.clip_by_value(tf.matrix_determinant(L), 1e-10, np.inf))
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/tensorflow/python/ops/gen_linalg_ops.py", line 273, in matrix_determinant
    result = _op_def_lib.apply_op("MatrixDeterminant", input=input, name=name)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
File "/home/user/bin/anaconda2/envs/condaenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): The determinant is not finite.
    [[Node: Model_likelihood/MatrixDeterminant = MatrixDeterminant[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Model_likelihood/add/_81)]]
    [[Node: Model_likelihood/MatrixDeterminant_3/_71 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/       task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_1280_Model_likelihood/MatrixDeterminant_3", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
$\endgroup$

closed as off-topic by Peter Flom Jan 31 at 10:51

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.