8
$\begingroup$

I have a question about clusters that I am contemplating to treat with a nonparametric mixture approach (I think). I am working on the explanation of human comportment.
Each row of my database contains:

  1. the ID of someone
  2. some parameters of the environment X (example: the temperature, the wind, etc.)
  3. a binary variable Y representing the person's reaction to the parameters (example: get sick or does not get sick because of the weather).

My idea (based on intuition and not on data) is that we can gather people in a finite number of groups so that in a group, people have the same reaction to temperature (some are easily sick, others are never sick...). In a given group, more formally, the law of Y conditional to the parameters X is the same.

I have no idea of the law of Y conditional to X. For the parameters X, I can do some hypothesis if necessary.

I would like to create some cluster of people "having more or less" the same reaction to parameter. Besides, I would like to predict the reaction of a given person to a given value of the parameters (even if this event has never happened in the database).

It seems to me that we can treat the problem like a nonparametric mixture model. As I don't have hypothesis on the conditional law of Y, I think I will have to create it with kernels method for instance. I have found this paper. Besides, it seems to me that, in this case, each row of observation $(X_i, Y_i)$ is not a simple realization of some random variable, but $X_i$ is a realization of a random variable, and $Y_i$ is a realization of a random variable conditional to $X_i$. I don't know if it makes a difference.

I have around 100000 rows. The vector $X_i$ has some discrete components, and others are continuous. I am wondering:

  • Is my approach correct?
  • Would you advise another point of view for this problem?

I would be very interested in any references about it.
Don't hesitate to ask me to reformulate the problem statement.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Since you have labelled data it is simply a supervised classification problem and what you need is a binary classifier. Which classifier works "best" is a matter of personal taste and the specific problem. Definitely, I would try first some standard options (SVMs, decision trees etc.) before trying anything more exotic. $\endgroup$
    – iliasfl
    Jul 6, 2014 at 9:28
  • $\begingroup$ Unless you are interested in the structure of the set, @iliasfl's comment is totally valid: classifier learning techniques will do well. Otherwise, you should use unsupervised learning. Before trying nonparametric models, you can try simpler EM algorithm for separating Gaussian mixture, or just k-means clustering, if you can estimate the number of clusters. Then, you may use its result for classification, or again classify raw data. $\endgroup$
    – Roman Shapovalov
    Aug 5, 2014 at 14:07

1 Answer 1

Reset to default
1
$\begingroup$

Answering to your point "Would you advise another point of view for this problem?", I would suggest that you actually have a look at your data. This can help you better plan what next steps to take. After all, the human eye-brain system is quite good in pattern recognition and you might be able to better decide upon the number of clusters, should you opt for an unsupervised clustering.

Accordingly, and since your data seems to be "high"-dimensional, you could try to perform a Principal Components Analysis (PCA) as this is a very quick analysis, especially for your dataset of 100k points. PCA, though, is not the only and not necessarily the most appropriate approach for dimensiona reduction with the goal of (2D/3D) visualization as it is a parametric, linear method. Your data may behave nonlinear though. I can suggest the dimension reduction toolbox for Matlab from Laurens van der Maaten which include a lot of different techniques. However, some of the techniques therein are inherently slow, so you might want to test them on subsampled data. A very recent and powerful nonparametric and nonlinear dimension reduction technique is BH-SNE which should also work for your dataset size, although it could take around 30 minutes to 1 hour depending on your available hardware. Since you are interested in the detection of clusters, BH-SNE might be a good choice as it (and it's "predecessor" t-SNE) has shown impressive performance in these regards on various datasets (s.a. the manuscript).

Finally, addressing your point on continuous/discrete data, this is something where I do not yet have experience how this influences the dimension reduction. Accordingly, you might want to try either discretizing the continuous variables or ignoring the (few?) discrete variables, if possible. Alternatively, you might want to take the binary variable (person's reaction) to color-code the points in the low-dimensional (2D/3D) visualization.

P.S. Performing a hierarchical clustering (linkage analysis) and looking at the resulting dendrogram is another way for creating a low-dimensional representation of you data which can help you better estimate if there are clusters and potentially also how many clusters there are.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.