2
$\begingroup$

I am working on creating a cost estimate for an order code, which is an order for medical procedures (category variable). I have estimated costs by patient encounter (visit to the doctor).

I am creating dummy variables for the order codes. I am then grouping the patient encounters so all the appropriate category variables are assigned.

I have estimates working ok when there are no overlaps in the dummy variables. When order codes are overlapping, they are being treated independently (this is actually not a bad result). So, if someone gets their arm and leg worked on, that's a different estimate than the arm and leg separately.

Is there any way to get dummy groups to interact with the individual dummy variables? So, arm orders have interactions with the arm + feet order estimates?

Below is what I have working so far:

import pandas as pd
from statsmodels.formula.api import ols

df = pd.DataFrame({
  'encounter' :[124,124,123,123,111,113,234,345,645,645]
, 'cost'      :[3000,3000,3300,3300,2500,2700,3000,4000,6000,6000]
, 'order_code':['foot','arm','arm','foot','arm','arm','hip','leg','hip','leg']})  

After the sample data is loaded into a dataframe, you can see encounter 124, 12 and 645 have two order_codes. These will get merged to the same encounter row later:

   encounter  cost order_code
0        124  3000       foot
1        124  3000        arm
2        123  3300        arm
3        123  3300       foot
4        111  2500        arm
5        113  2700        arm
6        234  3000        hip
7        345  4000        leg
8        645  6000        hip
9        645  6000        leg

I create the dummy variables from the order_codes, then add the encounter & cost data:

dummy = pd.get_dummies(df.order_code).join(df[['encounter','cost']])

Then, I merge the encounters together using the .max() function to make sure a "1" is present for each dummy variable.

group = dummy.groupby(by='encounter').max()

After the merge, the data looks like this. Notice encounters 123,124 and 645 both have multiple dummy values indicating multiple orders.

encounter  arm  foot  hip  leg  cost
111          1     0    0    0  2500
113          1     0    0    0  2700
123          1     1    0    0  3300
124          1     1    0    0  3000
234          0     0    1    0  3000
345          0     0    0    1  4000
645          0     0    1    1  6000

At this point, I apply a regression using all the dummy variables:

model = ols(formula='cost ~ arm + foot + hip + leg ',data=group).fit()

Here is the summary of that fit:

==============================================================================
Dep. Variable:                   cost   R-squared:                       0.993
Model:                            OLS   Adj. R-squared:                  0.978
Method:                 Least Squares   F-statistic:                     66.27
Date:                Wed, 03 Aug 2022   Prob (F-statistic):             0.0149
Time:                        15:51:14   Log-Likelihood:                -41.909
No. Observations:                   7   AIC:                             93.82
Df Residuals:                       2   BIC:                             93.55
Df Model:                           4
Covariance Type:            nonrobust
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept   1000.0000    312.250      3.203      0.085    -343.503    2343.503
arm         1600.0000    337.268      4.744      0.042     148.851    3051.149
foot         550.0000    180.278      3.051      0.093    -225.672    1325.672
hip         2000.0000    254.951      7.845      0.016     903.034    3096.966
leg         3000.0000    254.951     11.767      0.007    1903.034    4096.966
==============================================================================
Omnibus:                          nan   Durbin-Watson:                   2.385
Prob(Omnibus):                    nan   Jarque-Bera (JB):                0.287
Skew:                           0.000   Prob(JB):                        0.867
Kurtosis:                       2.009   Cond. No.                         10.0
==============================================================================

If I just run the data back thru the predict function i can get an idea what the cost estimates are looking like:

pred = model.predict(group).to_frame().join(group)

encounter estimate arm foot  hip  leg  cost
111        2600.0    1     0    0    0  2500
113        2600.0    1     0    0    0  2700
123        3150.0    1     1    0    0  3300
124        3150.0    1     1    0    0  3000
234        3000.0    0     0    1    0  3000
345        4000.0    0     0    0    1  4000
645        6000.0    0     0    1    1  6000

The estimates are working when the category variables are unique per encounter (encounters 111,113,234, and 345), they are also working when I have multiple categories per encounter (encounter 123,124 and 645). But, it would be really great if the individual dummy variables had some interaction with the dummy groups.

Here is the forecast just one dummy variable per encounter:

df_test = pd.DataFrame({'encounter' :[995,996,997,998]
                   , 'cost'     :[0,0,0,0]
                   , 'order_code':['foot','arm','hip','leg']}) 

When I run this, thru the predict function, results below:

pred_test = model.predict(group_test).to_frame().join(group_test)
print(pred_test)

encounter estimate  arm  foot  hip  leg  cost
995        1550.0    0     1    0    0     0
996        2600.0    1     0    0    0     0
997        3000.0    0     0    1    0     0
998        4000.0    0     0    0    1     0

Any suggestions on how to get the dummy groups (encounters with more than one dummy variable) and non-groups to influence each other?

Also, if there is a better way to tackle this problem, I am open to other suggestions.

Thanks!

$\endgroup$
2
  • $\begingroup$ IIUC, "arm", "foot", ... are your dummy variables. And you want them to have an interaction with the "dummy groups". But interaction (in the sense in which this notion is used in regression theory) exists between different variables, and I don't know what your "dummy group" variables are. So please clarify: What exactly do you mean by "dummy group" and what by "interaction"? $\endgroup$
    – frank
    Aug 4 at 9:51
  • $\begingroup$ Encounters 123 and 124 are acting as a "group" as the cost estimate is only using those two encounters. I believe they are being grouped in the regression because there are two dummy variables in those encounters. Is there a way that encounter 111 and 113 could also interact with that grouping? All the "arm" dummy variables should interact across all encounters. Its a bit tricky because the "foot" dummy is hiding in the grouping. $\endgroup$
    – Greg May
    Aug 4 at 16:25

1 Answer 1

0
$\begingroup$

IIUC, when you say "interaction", you are just referring to the fact that all the encounters that contain an arm treatment should have an influence on the cost estimate of an arm treatment. But this is actually already the case in your model.

In the comments you state "Encounters 123 and 124 are acting as a "group" as the cost estimate is only using those two encounters." However, the cost estimates of an arm treatment and of a food treatment are not only using those two encounters, they are using all encounters that contain an arm or a foot treatment. The model tries to find those treatment costs that satisfy the costs of all the encounters as well as possible (the total error will be minimized).

Your model presumes, that the treatment costs, when multiple treatments are done in the same encounter, simply add up. If that is indeed the case, you are all set.

Also, note that in regression theory, "interaction" means something different. E.g., you would have an interaction between arm treatment and foot treatment if the cost of the arm treatment would be different depending on whether it is combined with a foot treatment, i.e. the cost of an encounter with arm and foot treatment would not be just the sum of an arm and a foot treatment.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.