0
$\begingroup$

I have a binary dependent variable that I am interested in predicting with historical data. I have 19.000 observations from 1900 and onwards, and I would like to study the relationship between year/decade and the DV. Normally, in my field, what people do is that they code for the decade (1900, 1910, 1920, etc.) without taking into consideration variation within the decades. Also, coding by decade seems a bit arbitrary to me, since it is just a construction of time which has no validity in reality.

Taking that into account, I would like to find another way of grouping historical data than "decade". More specifically, I would like to identify meaningful clusters where the data points group dependending on the value of the DV. In that way, I would theoretically be able to find groups like "1900-1933; 1934-1978; 1979-2019", etc. These different clusters would then constitute a new predictor that I could enter into a GLM as a categorical factor. So the assigned group for each observations needs to be saved as a new variable in the frame.

Another option would of course be to introduce "year" as a numerical factor. However, I am interested in exploring the above explained option.

Does anyone know what type of cluster analysis would be required to be conducted?

Thank you in advance. I am using R for this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Define"meaningful".

The are methods to detect abrupt change in mean. You could use these to chunk your data.

Basseville, Michèle, and Igor V. Nikiforov. Detection of abrupt changes: theory and application. Vol. 104. Englewood Cliffs: Prentice Hall, 1993.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ I think you would like to add this as a comment. $\endgroup$
    – usεr11852
    Commented Mar 2, 2019 at 13:40
  • $\begingroup$ The recommendation to use the usual and we'll known abrupt-change-in-mean methods is more than just a comment. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Mar 2, 2019 at 23:04
  • $\begingroup$ "meaningful" in a sense that the groupings are based on actual observations rather than a priori groupings, like "decade". The different chunks would (I'm guessing) be clustered according to degree of homogeneity. Would it make sense to measure mean when the DV is categorical? $\endgroup$
    – mmarttin
    Commented Mar 3, 2019 at 14:05
  • $\begingroup$ Well, a standard quantile approach would also be data based... Usually, I would consider a decade a "meaningful" grouping (as it has a defined meaning), whereas a data-based grouping could be quite arbitrary (1923-1942). $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Mar 3, 2019 at 16:49
  • $\begingroup$ Thank you for your comment. Sure, I understand your point. However, what I mean is that "decade" doesn't really mean anything in relation to your data - you could for example have significant heterogeneity within a decade, and then that grouping no longer makes sense - right? I would like some way of finding a data-based grouping rather than a priori. $\endgroup$
    – mmarttin
    Commented Mar 4, 2019 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.