I have a panel dataset that contains information on households and their purchases in specific weeks and even minutes of a week. For example, a household would purchase different brands of beer and record those purchases. If they buy 3 different brands of beer in one purchase, they would record each brand separately. These three transactions will definitely show same household id, same week, and same minute but different UPC codes. In fact, this is how many replicates I have in my dataset:

duplicates report panid week minute

copies | observations       surplus
    1 |        12350             0
    2 |         3728          1864
    3 |          768           512
    4 |          240           180
    5 |           65            52
    6 |           36            30
    8 |            8             7
   10 |           10             9

This is how it would look:

panid    week   minute
3817767  1635   799
3817767  1635   799
3817767  1635   799

At first, I wanted to combine both week and minute into one variable totalminute by calculating the total amount of minutes passed since period 0. For example, January 1, 2011 00:00 will count as munite 0, and December 31, 2011 23:59 will be minute 525949.

Now I realize that this will not be a great idea, I will still have repeated indicator-week combinations. I stumbled upon a couple of sources including this one answered by Nick Cox:

Unique time variable panel regression fixed effect

But so far, cannot see a problem similar to mine. Aggregation on the household level by week doesn't sound appealing because I am trying to build a choice model that is interested in each unique purchase.

If you can provide me any sense of direction on how to create unique combinations of indicator-time variables, I will greatly appreciate it.


Can I possibly create the variable n

bysort panid week minute: gen n=_n

and maybe combine it with each week observation, thus creating a unique time variable?


This is not going to be a perfect answer but hopefully it helps you forward. I do not think that creating a unique time variable is the best way forward. You have to think about what makes most sense from a research perspective. What are you really interested in.

I think there are a few options to consider: 1) Use a fixed effect regression and group around households (iis panid ; xtreg y xB , fe). You can use a random effect regression as well and test which one fits best via Hausman procedure (run est store fixed and est store random after both regressions and then use hausman fixed random and check. You could then add dummies for weeks and potentially even different dummies for time (if you assume for instance that purchases at different moments in the day will reveal different behaviour). 2) Do the same thing but use your weeks as a grouping variable iis week . This is not conventional but surrounds the problem of having multiple observations at the same time. Then all the variation in purchases behaviour need be explained by the information you have about households.

In addition, you can cluster the errors around year (panid) respectively (although this might fail depending on your data nesting. You can potentially make your minutes more categorical (perhaps 1 per hour) and cluster according to week _ hour (see link1 or link2 for more on clustering in Stata).

Depending on the number of observations you have in total, Stata might not be ideal to do this (for really big data with multiple clusters). Then you could consider R and R package(lfe)

Good luck,



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.