# Linear regression with independent variables with varying proportions

I am looking to do a linear regression on two independent variables that will be present in varying proportions.

For example trying to do a linear regression on $Y$ which is payment behavior (payback rate) of customers based on the the quality (let's say Gini coefficient) of the new and existing customer credit scores ($X_1$ and $X_2$, respectively) adjusted for the proportion of new and existing customers in the sample.

Existing customers will be present in proportion $p$ and new customers in proportion $1-p = q$.

$Y$, payback rate is the percentage of total customers who pay back. It could be expressed as the weighted average $Y = Y_1q + Y_2p$ where $Y_i$ is the payback rate of new/existing customers.

In general more new customers, $q$, has a negative effect. Better scoring ($X_1, X_2$) and more existing customers p have a positive effect.

What is a good way to model this?

Would something like the following be a good solution trying to use $p$ and $q$ as some sort of interaction effect?

$Y = X_1+X_2+\frac{X_1}{q}+X_2 p$

Would it be better to include p and q as variables themselves as well?

• It would help to clarify by defining explicitly and in mathematical terms what $Y$ is. "Payback rate" is a very general term. Is it measured as a ratio of monetary values? Does it count instances of payments-non payments? Is it a weighted average of the behavior of old and new customers? And if yes, what are the weights used? Etc. How one models the RHS has obviously a great deal to do with what exactly does this RHS attempt to explain (the LHS of the regression specification). Otherwise, it would be blind mechanical search for a good fit. – Alecos Papadopoulos Oct 14 '13 at 16:34
• Good point. Y now defined more clearly above. – Dirk Calloway Oct 15 '13 at 8:16

From what I can understand, the real-world phenomenon under study can be described as follows:
There are at every period $N_t$ customers which are divided given some criterion into "existing" and "new" (think why the categorization criterion is not necessarily obvious).

For each of these two subgroups, we define the "payback rate = percentage of customers of this subgroup who pay back". For new customers we denote this payback rate $Y_1$ and for existing customers we denote this payback rate $Y_2$. We also have as possible explanatory variables the credit scores for these customers. I presume that $X_1$ symbolizes the average credit score of new customers, and $X_2$ the average credit scores of existing customers.

Now, these payback rates should be examined separately, before attempting to build a model for their weighted average.

We may assume therefore that $$Y_{1t}= a_1 + b_1X_{1t} + u_{1t} \qquad [1]$$ and $$Y_{2t}= a_2 + b_2X_{2t} + u_{2t} \qquad [2]$$

with $t=1,...,T$ being the length of the time series, and the two error terms assumed white noises, independent of each other, and independent of the regressors.

What we want is to estimate the weighted average pay back rate. Denoting $p_t$ the existing customers as a percentage of the customer base $N_t$, this weighted average payback rate is exactly defined as

$$Y_t = (1-p_t)Y_{1t} + p_tY_{2t} \qquad [3]$$

Relation [3] is a mathematical identity. We turn it into a causal/associative/covariance relationship by inserting into it equations $[1]$ and $[2]$ that refelct theoretical/behavioral assumptions:

$$\{[1],\,[2],\, [3]\} \Rightarrow Y_t = (1-p_t)\left(a_1 + b_1X_{1t} + u_{1t}\right) + p_t\left(a_2 + b_2X_{2t} + u_{2t}\right)$$

$$\Rightarrow Y_t = \left[(1-p_t)a_1 + p_ta_2\right] + (1-p_t)b_1X_{1t} + p_tb_2X_{2t} + (1-p_t)u_{1t} + p_tu_{2t}$$

$$\Rightarrow Y_t = a_1 + (a_2-a_1)p_t + b_1X_{1t}^* + b_2X_{2t}^* + \varepsilon_t \qquad [4]$$

with

$X_{1t}^* = (1-p_t)X_{1t}$ and $X_{2t}^*= p_tX_{2t}$ sub-group credit scores weighted by the relative size of each sub-group,

but most importantly with

$\varepsilon_t = (1-p_t)u_{1t} + p_tu_{2t}$

This means that the error term is contemporaneously correlated with all three regressors.

Denoting by $\mathbf X$ the regressor matrix containing the time series for $\left(p,X_{1}^*,X_{2}^*\right)$ The conditional moments of $\varepsilon_t$ are

$$E(\varepsilon_t\mid \mathbf X) =E((1-p_t)u_{1t} + p_tu_{2t}\mid \mathbf X) = (1-p_t)E(u_{1t} \mid \mathbf X) + p_tE(u_{2t}\mid \mathbf X) = 0$$

since the $u$-errors are independent of $\mathbf X$ and white noises. Also
$$\operatorname {Var}(\varepsilon_t\mid \mathbf X) = E(\varepsilon_t^2\mid \mathbf X) = (1-p_t)^2E\left(u_{1t}^2\mid \mathbf X\right) + p_t^2E\left(u_{2t}^2\mid \mathbf X\right)$$ $$= (1-p_t)^2Eu_{1t}^2 + p_t^2Eu_{2t}^2=(1-p_t)^2\sigma_1^2 + p_t^2\sigma_2^2$$

i.e. the error term is conditionally heteroskedastic, with the conditional variance depending on the regressor $p_t$.

We have arrived at the regression specification $[4]$ by recognizing that the dependent variable is necessarily constructed as a function of the regressors, and so our behavioral/association assumptions should be "placed" one step earlier (at sub-group level). From this, the interaction between the regressors emerged naturally. But also, we ended up with endogenous regressors and a heteroskedastic error term, the variance of which changes in each time period and is a function of one of the regressors. Finally, one should think that, if the model is to be used for prediction, perhaps the $p_t$ should be modeled as an autoregressive scheme, since it is difficult to think that the existing/new customers allocation can exhibit wide variations from one period to the next.

Concluding, this is your model, or at least, a model that is consistent with the real-world phenomenon under study. As you can see, you have much more serious issues to deal with than just ad hoc ways to represent the interaction between the regressors...

• Interesting hadn't thought of that with the variance. New customers have a higher variance so more news means not only worse performance but higher variance and it is definitely heteroskedastic... – Dirk Calloway Oct 16 '13 at 15:02

I would stick with $p$ only, as $q$ does not add any information on top of $p$. I would add interaction terms between $X_1$ and $p$ and $X_2$ and $p$ and then include the main effects of both $X_1$, $X_2$ and $p$. So:

$Y =\beta_0 + \underbrace{\beta_1 X_1 + \beta_2 X_2 + \beta_3 p}_{\textrm{main effects}} + \underbrace{\beta_4 X_1 p + \beta_5 X_2 p}_{\textrm{interactions}} + \varepsilon$