Skip to main content
edited title
Link
user88
user88

Maximum Entropy Inferenceentropy inference for k-Meansmeans clustering

added 216 characters in body
Source Link
blubb
  • 2.7k
  • 3
  • 24
  • 29

I am taking a course on Maximum Entropy Inference (MEI), where its application to k-Means was discussed. I am confused about the problem setting.

As far as I understand, our goal is to find clustering solutions $c(\cdot)$ with small expected cost $\mathbb{E}[R^{km}(c)] \le R_{max}$. We are therefore interested in sampling from a distribution $p^*(c)$ over these solutions where good solutions have high probability and bad solutions have low probability. In order to be as 'agnostic' as possible, we ask for the distribution which maximizes the entropy, and we thus get:

$$ p^*(c) = \arg \max_{p(c)} {H(c)} \quad \text{ s.t. } \mathbb{E}_p[R^{km}(c)] \le R_{max} $$

Obviously, $c(\cdot)$ are random variables in this scenario. But what is the underlying proability space? If $p^*(c')$ is large, it tells that $c$ is a good clustering solution, but... for what problem? Is this for a specific dataset $\mathcal{X}$ or for a large variety of such datasets? In either case, where does this or do these datasets play into the above equations?

EDIT: I hope I am not being too vague here. Unfortunately I can't express myself very clearly as I don't understand the topic that well. However, I might be able to answer specific questions less ambiguously.

I am taking a course on Maximum Entropy Inference (MEI), where its application to k-Means was discussed. I am confused about the problem setting.

As far as I understand, our goal is to find clustering solutions $c(\cdot)$ with small expected cost $\mathbb{E}[R^{km}(c)] \le R_{max}$. We are therefore interested in sampling from a distribution $p^*(c)$ over these solutions where good solutions have high probability and bad solutions have low probability. In order to be as 'agnostic' as possible, we ask for the distribution which maximizes the entropy, and we thus get:

$$ p^*(c) = \arg \max_{p(c)} {H(c)} \quad \text{ s.t. } \mathbb{E}_p[R^{km}(c)] \le R_{max} $$

Obviously, $c(\cdot)$ are random variables in this scenario. But what is the underlying proability space? If $p^*(c')$ is large, it tells that $c$ is a good clustering solution, but... for what problem? Is this for a specific dataset $\mathcal{X}$ or for a large variety of such datasets? In either case, where does this or do these datasets play into the above equations?

I am taking a course on Maximum Entropy Inference (MEI), where its application to k-Means was discussed. I am confused about the problem setting.

As far as I understand, our goal is to find clustering solutions $c(\cdot)$ with small expected cost $\mathbb{E}[R^{km}(c)] \le R_{max}$. We are therefore interested in sampling from a distribution $p^*(c)$ over these solutions where good solutions have high probability and bad solutions have low probability. In order to be as 'agnostic' as possible, we ask for the distribution which maximizes the entropy, and we thus get:

$$ p^*(c) = \arg \max_{p(c)} {H(c)} \quad \text{ s.t. } \mathbb{E}_p[R^{km}(c)] \le R_{max} $$

Obviously, $c(\cdot)$ are random variables in this scenario. But what is the underlying proability space? If $p^*(c')$ is large, it tells that $c$ is a good clustering solution, but... for what problem? Is this for a specific dataset $\mathcal{X}$ or for a large variety of such datasets? In either case, where does this or do these datasets play into the above equations?

EDIT: I hope I am not being too vague here. Unfortunately I can't express myself very clearly as I don't understand the topic that well. However, I might be able to answer specific questions less ambiguously.

Source Link
blubb
  • 2.7k
  • 3
  • 24
  • 29

Maximum Entropy Inference for k-Means

I am taking a course on Maximum Entropy Inference (MEI), where its application to k-Means was discussed. I am confused about the problem setting.

As far as I understand, our goal is to find clustering solutions $c(\cdot)$ with small expected cost $\mathbb{E}[R^{km}(c)] \le R_{max}$. We are therefore interested in sampling from a distribution $p^*(c)$ over these solutions where good solutions have high probability and bad solutions have low probability. In order to be as 'agnostic' as possible, we ask for the distribution which maximizes the entropy, and we thus get:

$$ p^*(c) = \arg \max_{p(c)} {H(c)} \quad \text{ s.t. } \mathbb{E}_p[R^{km}(c)] \le R_{max} $$

Obviously, $c(\cdot)$ are random variables in this scenario. But what is the underlying proability space? If $p^*(c')$ is large, it tells that $c$ is a good clustering solution, but... for what problem? Is this for a specific dataset $\mathcal{X}$ or for a large variety of such datasets? In either case, where does this or do these datasets play into the above equations?