4
$\begingroup$

I was looking at a regression problem I have, and I'm using the R(squared) metric to assess my model, in addition to other metrics. I'm aware that R(squared) will always increase as the number of features increase/add more features, which I'm also using the R(squared) adjusted metric to account for this change.

Using the following formula, I'm having doubts about the actual value of p.

enter image description here

If I have a number of independent variables (a mix between numerical and categorical), should the value of p be based on the number of variables after processing categorical features or not?

Lets say I have 3 variables: Num_1, Cat_1, Cat_2. And Cat_1 is a binary categorical variable, and Cat_2 has 3 unique values. Would the value of p be 4? Since Cat_2 gets encoded into 2 variables. Or is it 3? Not accounting encoding.

$\endgroup$
2
  • 1
    $\begingroup$ It's not clear from your description how you're representing your data to the model. How many coefficients are you estimating? If both cat_1 and cat_2 are encoded using 1-hot encoding, then $p = 1 + 2 + 3$. But if cat_1 is binary encoded and cat_2 is dummy-encoded, then $p = 1 + 1 + 2$ (including num_1). If cat_3 has only 3 values, but those values are numerical (e.g. -0.5, 1.2, 101.7), then cat_3 only estimates one coefficient, so cat_3 only counts as 1 for computing $p$. $\endgroup$
    – Sycorax
    Commented Feb 18, 2022 at 15:10
  • $\begingroup$ I think it's dummy-encoded. I was going to mention the dummy variable trap, but did not find necessary as per OP's examples. But, yes, I believe it should be clarified better. $\endgroup$
    – gunes
    Commented Feb 18, 2022 at 15:15

1 Answer 1

5
$\begingroup$

It's the number of independent variables entering into the regression model.; therefore it's the number after any preprocessing step performed. In this case it is 4, assuming you're using dummy-encoding as per your examples suggest.

$\endgroup$
1
  • 1
    $\begingroup$ +1 Phrased informally, the regression model doesn’t know or care how you wound up with the features, just that there are parameters to estimate. $\endgroup$
    – Dave
    Commented Feb 18, 2022 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.