1
$\begingroup$

Sometimes I have numerical columns that are composed of two unique values. For example, a value from the set $\{0.1, 5.4\}$ in every cell, or $\{-1, 0\}$ in every cell. I typically scale these columns with the rest of the numerical columns.

However, before my scaling step I perform an encoding step where I look for categorical columns with two unique values and encode with a label encoder.

This step could easily capture my two-unique-value numerical columns too, which would save me a scaling step.

However, I'm worried I'll lose important information. Can it be safe to label encode numerical columns with two unique values only?

$\endgroup$
7
  • 4
    $\begingroup$ Please provide more information about the context of this problem. First, ¿what do you mean by binary numerical data? Second, please explain what you mean about a column having 2 unique values...it seems all columns would have 2 (and only 2) unique values (if the data is binary). $\endgroup$
    – Gregg H
    Apr 17, 2023 at 11:35
  • 1
    $\begingroup$ "an encoding step where I look for categorical column with 2 unique values and encode with a label encoder." You mean that any regressor that has only two values is treated as a dummy variable? That's what they effectively are. What was your exact question? $\endgroup$ Apr 17, 2023 at 18:46
  • $\begingroup$ @SextusEmpiricus What do you mean by regressor? Do you mean column/feature/variable? Or are you talking about a model? I believe the question has a simple answer, there is no hidden meaning. I'd like to know if there's an information loss when you label encode a numerical column that is composed of two unique values (only). My current thinking is that the answer is no, but I want to check that before I go ahead and do it! $\endgroup$
    – Connor
    Apr 18, 2023 at 12:47
  • 1
    $\begingroup$ Yes I mean a 'variable', the columns in the regressor matrix. $\endgroup$ Apr 18, 2023 at 12:50
  • $\begingroup$ @SextusEmpiricus Then yes we're absolutely talking about the same thing! What do you think needs updating to get an answer to this question? It's not meant to be a deep question, I think it could be answered very easily. But I haven't been able to find a clear answer, perhaps because it's a silly question, but I'd like that confirmed one way or the other! $\endgroup$
    – Connor
    Apr 18, 2023 at 15:09

1 Answer 1

0
$\begingroup$

It's possible that you can lose information label encoding a column like this without knowing what the values represent and what they're going to be used for.

Assuming you're columns have two numerical variables, there are two general situations to consider from a modelling perspective.

1. Models that don't implicitly scale the data

In this case you have to take into account exactly what affect your scaling and encoding steps have. If they change the data in different ways the model has no way of changing them to get the same final end point.

Inevitably, this will give you different final models.

2. Models that do implicitly scale the data

In this case encoding and scaling steps will have exactly the same effect, because whilst information on the magnitude is lost, the relationship is preserved.

The model's implicit scaling will bring them to roughly the same endpoint. Therefore, encoding or scaling are equally safe in this situation.

Takeaway

In your case, as you already have the scaling step set up, I would continue to use that. If you really want to encode, try both and check it makes no difference. That's the best way to be sure.

$\endgroup$
10
  • $\begingroup$ You absolutely do lose information by scaling data! Moreover, it's unclear what "scaling" might mean for pairs of numerical values, whatever they might represent. $\endgroup$
    – whuber
    Apr 19, 2023 at 13:15
  • $\begingroup$ Do you lose meaningful information? Will it affect the model? From what I've read, it doesn't. $\endgroup$
    – Connor
    Apr 19, 2023 at 19:20
  • 2
    $\begingroup$ Ones that are nonlinear in the regressors: the first class that I mentioned. The point is that in linear models, the scaling factor can be absorbed automatically into the coefficients. That won't work for nonlinear models. $\endgroup$
    – whuber
    Apr 19, 2023 at 19:42
  • 1
    $\begingroup$ Think about how those models work and apply the criterion I just named: if the scaling factor can automatically be accommodated, then there's no problem. When it can't, you have to be careful. $\endgroup$
    – whuber
    Apr 19, 2023 at 19:45
  • 1
    $\begingroup$ Ahhh interesting! I hadn't considered that. So that means when scale can be accommodated in the weight encoding and scaling are the same. But when the model can't accommodate the scaling they'll have different effects. $\endgroup$
    – Connor
    Apr 19, 2023 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.