0
$\begingroup$

I am dealing with a large data set (~100000 observations, roughly 13 variables). Most variables are of categorical nature and are stable across different observations concerning the same individual/entity, e.g. social class or income, however some are not.

My Problem is rooted in the fact that there is a varying number of observations per individual.. so there might be 3 observations belonging to the same individual or there might be 17 observations.. Therefore, I want to aggregate all observations for each individual which is rather unproblematic for variables like social class (in this data set no individual is changing social classes). So, the aggregated social class variable will be of the same categories as the original one.

However, there is one variable "brand loyality" which takes on the values of 0 (=brand loyal) or 1 (=not brand loyal). Each observation describes a purchase of a certain type of homogenous good. If 2 consecutive purchases come frome the same brand the value is 0, otherwise 1. The variable as a whole merely checks what brand was bought last (so it is kind of a lagged variable somehow) Aggregating this variable yields a number between 0 and 1 and is not of categorical nature anymore (e.g. 0.34)

Nevertheless, I believe this variable, even though not categorical anymore, conveys some information, e.g. if the variable would take on a value of 0.2 then the individual is on average rather brand loyal. If the value would be 0.8 then I would say the individual is on average by trend not loyal.

Is it good statistical practice (probably not :p) to include this aggregated variable in my regression model? And if so, I would like to know some more about the theoretical idea behind this (Why is it ok?/Why is it not ok?)

I was just wondering because this variable is highly significant in my linear model and I don't know why - at least not from the statistical point of view. From the theoretical point of view/according to my intuition this makes a lot of sense.

$\endgroup$

1 Answer 1

2
$\begingroup$

I'm not sure if I'm understanding correctly, but you aren't doing multilevel modeling, right? I believe that multilevel modeling could handle the aggregation by parsing the within-person variance and the between-person variance, while the the variables that don't vary within-person should be simply be treated as between level variables (don't know what repercussions would the binary nature of the variables would have). I think that this would adjust for the non-independence of observations that would occur by adding repeated measures of a non-varying state, the underestimation of standard errors. I've merely heard about multilevel modeling though, you could read if it can be of help.

If the three observations are not being considered as longitudinal in character, perhaps you could model a latent variable using these three observations as indicators (with the proper technique, e.g., wlsmv or irt models, the binary measurement would be no problem as lots of literature will confirm). A latent variable could be considered better than an average score in that it's based on the covariance among the indicators and tries to eliminate "measurement error", and perhaps model fit could be evaluated to test if it makes sense to try to aggregate them. Sorry if that's not helpful.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. I did not do a multilevel analysis, but due to your comment I did some research on multilevel analysis and this is probably what I should have done :p It did not occur to me that "Observation concernting individual i" (level 1) and "individual i" (level 2) can be considered as two levels (it is not as obvious as pupil and class for instance). $\endgroup$
    – avey
    Nov 22, 2017 at 11:00
  • $\begingroup$ I have to admit that I merley calculated the means/average values for each individual and treated the resulting data set as enirely new data set, which is probably wrong since it neglects the within-person variance. Is there and easy and convenient way to do a multi-level analysis in R? If you could provide some sample code I would be deeply grateful (since I have never done one before.) Tanks! $\endgroup$
    – avey
    Nov 22, 2017 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.