Timeline for A seeming paradox regarding estimation of the number of buttons
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 25 at 12:18 | comment | added | Sextus Empiricus | I might try your algorithm. But I fail to see how it is a Gibbs sampling scheme (it could be however a way to find the maximum posterior estimate), and also the relationship with the hypotheses about $N$ are not clear. How are the hypotheses about the values of $N$ incorporated? An MAP estimate for the vector $\theta$ is also not the same as the posterior distribution of $N$ (which is marginal and needs to integrate out the possibilities for the values of $\mu_i$). | |
| Feb 25 at 12:06 | comment | added | Spätzle | You only sample from the prior for the initial value, nothing more. I suggest at this point that you experiment a bit with numeric approximations and algorithms because it is pretty much covered in every course | |
| Feb 25 at 11:49 | comment | added | Sextus Empiricus | It is still unclear to me how the distinction for different hypotheses for the number of clusters $N$ is made. The sampling from the distribution of $\pi$ may likely contain non-zero sized clusters because different $\mu_i$ can be very close together. Mixing more clusters then there truly are will just lead to overfitting, and not to clusters of zero size. | |
| Feb 25 at 11:46 | comment | added | Sextus Empiricus | This 5-th step might be instead sampling from a prior distribution for the $\mu_i$ and the prior for a next step is the posterior from a previous step? | |
| Feb 25 at 11:30 | comment | added | Sextus Empiricus | How are $\mu_i^{(k+1)}$ determined of $N_i^{k} = 0$? | |
| Feb 25 at 11:27 | comment | added | Sextus Empiricus | A new $\mu_i^{(k+1)}$ is based on an average of the $d_j$ that are assigned to the $i$-th group. But this doesn't sample the $\mu_i$ from the entire space $\mu_i \in [1,10^6]$? Say if we have $k=2$ then only three potential values of $\mu_i$ are sampled, namely $d_1$, $\frac{d_1+d_2}{2}$ and $d_2$. | |
| Feb 25 at 11:21 | comment | added | Spätzle | This is described in step 5. The only time you need to explicitly set values for the expectations is for $\mu^{(0)}_i$, they could be randomly sampled from the prior or chosen according to some beliefs of the OP. I hate to say it but choosing initial values for such tasks is indeed an art and relies a lot on experience. | |
| Feb 25 at 11:20 | comment | added | Sextus Empiricus | "What happens eventually is that if the actual number of clusters is smaller that what we've initialized to, we'll get those clusters with 0 members and close" Why would those clusters get zero members, instead of becoming a non-zero cluster with just a same or similar mean $\mu_i$ as another mean $\mu_j$? | |
| Feb 25 at 11:16 | comment | added | Sextus Empiricus | How are the $\mu_i^{(k)}$ for the $k$-th step computed? | |
| Feb 25 at 11:15 | history | edited | Spätzle | CC BY-SA 4.0 |
added 2 characters in body
|
| Feb 25 at 11:14 | comment | added | Spätzle | The samples are in 1D, as defined "when pressed, the computer samples a real number from $\mathcal{N}(\mu_i,\sigma=1)$", hence $D_j\in\mathbb{R}^d$ with $d=1$. Computational complexity of Gibbs depends less on the number of candidate clusters and more on the data dimensionality ($d$) and the need to estimate the variance structure. As all variances are given and $d=1$, the whole process is very cheap computationally. If it were something like $D_j\sim\mathcal{N}(\mu_i,\Sigma_i)$ with $rank(\Sigma_i)=d>2$ - that was indeed costly. | |
| Feb 25 at 10:58 | history | edited | Spätzle | CC BY-SA 4.0 |
typos
|
| Feb 25 at 10:57 | comment | added | Spätzle | Nope, this is data with $d=1$ and known variance, should take at most a few seconds in R/python. It would have been far more complex if the data were not unidimensional. The number of clusters can be enlarged at will because it only has little effect (each iteration consists of simple math and RNG, nothing expensive such as matrix inverting) | |
| Feb 25 at 10:07 | history | answered | Spätzle | CC BY-SA 4.0 |