0
$\begingroup$

I have a dataset where rows are individuals and I want to measure the magnitude of the change in the distribution of their spending from year to year.

So for Person A:

  • T1 his values are: 30% A 30% B and 40% C

  • T2 his values are: 90% A 5% B and 5% C

I want to create a measure that will indicate the magnitude of the change that takes into account the change in proportion for each category.


EDIT Added more details below:

  • I have data for around 80K observations and 15 time points for each observation. Essentially I want to identify meaningful deviations in their spending behavior. So, to start, I figured I would need calculate the change from t1 to t2. Later, I except I will have to look at longer periods, but for right now I wanted to keep it as simple as possible

  • I'm using this measure as a way to gauge an individuals priorities in terms of how they allocate their money. So if someone who has allocated just 10% of their spending on Category A in T1 but then it becomes 90% in T2. This is more meaningful to me then a person to goes from 70% to 90% during the same time period. I know I could look at simple percent change for each category, but I wasn't sure if there was a more robust way to identify how large the change in behavior is across all of the categories

$\endgroup$
  • $\begingroup$ Without knowing what would count as a good measure for your scientific question this is very broad. Can you expand on what properties you are aiming for? $\endgroup$ – mdewey Nov 1 '16 at 16:54
  • 1
    $\begingroup$ How many years of data do you have? How many individuals are there? By analogy to time series of normal data, there can be a great deal of volatility around a relatively stable trend & little volatility around a rapidly changing trend. Which are you looking for here? W/ 3 categories, the data are multidimensional which means there are different kinds of changes that can occur. Can you specify what type might interest you? (Eg, if you were interested in A vs ~A, then [30, 20, 50] = [30, 60, 10].) $\endgroup$ – gung - Reinstate Monica Nov 1 '16 at 18:13
  • $\begingroup$ @mdewey I have data for around 80K observations and 15 time points for each observation. Essentially I want to identify meaningful deviations in their spending behavior. So, to start, I figured I would need calculate the change from t1 to t2. Later, I except I will have to look at longer periods, but for right now I wanted to keep it as simple as possible. $\endgroup$ – grad_student Nov 2 '16 at 17:41
  • $\begingroup$ @gung I'm using this measure as a way to gauge an individuals priorities in terms of how they allocate their money. So if someone who has allocated just 10% of their spending on Category A in T1 but then it becomes 90% in T2. This is more meaningful to me then a person to goes from 70% to 90% during the same time period. I know I could look at simple percent change for each category, but I wasn't sure if there was a more robust way to identify how large the change in behavior is across all of the categories. $\endgroup$ – grad_student Nov 2 '16 at 17:44
1
$\begingroup$

For each category ; for each person .... develop a robust ARIMA model for their annual absolute spending incorporating unknown deterministic structure (pulses/level shifts/local time trends) . Now assess if there is a pulse in the most recent period. If there is a pulse then flag this spending as exceptional. A caveat .. if their annual income is changing one has to possibly analyze the data conditionally upon their income thus a transfer function model would be in order.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ The strong heteroscedasticity and perfect correlation among the three series suggest the approach you recommend is likely to fail. Predictions won't sum to 100% and extremely small or large values will get more weight than they deserve. $\endgroup$ – whuber Nov 1 '16 at 21:30
  • $\begingroup$ I had suggested (meant) using the observed spending ( observed spending) not the percent change. $\endgroup$ – IrishStat Nov 1 '16 at 21:33
  • 1
    $\begingroup$ Thank you for the clarification--I had overlooked that distinction, which you clearly expressed as "annual absolute spending." $\endgroup$ – whuber Nov 1 '16 at 21:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.