I have a dataset that has 12 explanatory variables for every 1 observation.
I hypothesize that the data is generated by underlying process which undergoes several different phases, and that the 12 explanatory variables would somehow help in identifying clusters.
I want to write a machine learning algorithm that can:
At first pass, identify the clusters within the data
The data is dynamic in that new (additional) data is generated periodically. After classification (step 1 above), I want to be able to correctly label (i.e. classify) any new data not previously seen into one of the previously identified classes/clusters (or fail gracefully).
I assume that
Y(c) ~ X(c) + error
- Y(c) is a nx1 vector of observations belonging to cluster C
- X(c) is a nx12 vector of explanatory factors that 'belong' to cluster C
Observations in different clusters will differ from each other by having different distribution shapes. That is to say observations WITHIN a cluster will have a different distribution shape compared to observations from another cluster.
I am relatively new to machine learning, and would like some guidance on how to implement such an algorithm (or perhaps one already exists?)
I would be particularly interested in finding out how to 'classify' observations based on determining the shape of the empirical distribution of the observed data.
Last but not the least, I would also appreciate some advice on whether to implement this in R, or Octave (and why).