Aim: I want to determine what the drivers are of defaulted amounts on student loans. I also want to be able to project defaulted amounts into the near-term future (1-3 months from last date in my dataset [Sept 2020], for example).

Model: $$Defaulted Amount_{it} = \alpha_1 \mathrm{Int rate_{it}} + \alpha_2 \mathrm{Hasdefaulted_{it}}$$

where all the explanatory variables are categorical.

Raw Data: I have monthly data from 01/01/19 to 09/01/20, and a total of 5 million rows of data.

        Date          int_rate             \
0     01/01/19        4.6-5.0         
1     02/01/19        4.6-5.0       
2     02/01/19        4.1-4.5      
3     02/01/19        4.1-4.5           

  has_def     Sum_def_amount    
0    0           141,904.57         
1    1           141,657.98         
2    1           142,357.18          
3    0           133,137.11      

Panel regression: I have this info at the loan-level and across time and so I run a Panel regression in Python. I use the Fama Macbeth regression Model for the panel regression. In this model, the first step is to run the regression for each period cross-sectionally, i.e. pool N individual loans together for a given period t. And this is done for each period, t=1,...T. So in total T regressions are run. The average across the time series of coefficients is taken as the final coefficients of each independent variable. However, this description of the model is for continuous independents and I'm not too sure how to pool the loans together given that I have categorical independents.

How I pool the data: Essentially, I 1) group the loan level information by all the explanatory variables (which are categorical) and then 2) run a regression on that grouped data. Rather than pooling it across just Date (as is done in the answer link, for continuous independents), I create groups with similar characteristics (like the same int_rate, etc) and run the regression on these groupings. To my understanding, to capture the relative effects of each of these categorical explanatory variables I need to group in this manner. If these were continuous explanatory variables I could just pool the data by Date and run the regressions for each Date. I run this regression for each subset of data associated with a particular Date (please see the Subest data below) and eventually take the mean of the coefficients as the final models coefficients.

The regression (ols in Pythons statsmodels) is run on a set of data similar to the subset below, for each associated Date. The next subset of data that is passed to ols would have similar int_rate and has_def data but would be for the Date = 2019-02-01. The mean across the coefficients associated with each regression will be the final coefficients.

Data Subset:

         Date  int_rate      has_def  Sum_def_amount
0    2019-01-01  > 5.0         1  65417923.37
1    2019-01-01  > 5.0         0  24258938.16
2    2019-01-01  <=4.0         1  14564732.24
3    2019-01-01  <=4.0         0   6522590.22
4    2019-01-01   4.1-4.5         1   7813393.75
5    2019-01-01   4.1-4.5         0   1256053.98

Question: The problem here is that the standard OLS regression is run on each of these above subsets, where all of the data is associated with a particular date (which defeats the purpose of the pooling). How can I redo the pooling part to prevent this from happening?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.