1
$\begingroup$

I have measurements (survey data) from two different sources. The scaling for one discrete attribute is different.

The values of the attribute in the first survey are: 1, 2, 3

The values of the attribute in the second survey are: 1, 2, 3, 4, 5, 6, 7

Normalized: 1/3, 2/3, 1 and 1/7, 2/7 ... 1

The preprocessed data is then used for machine learning. But the algorithms keep coming up with rules like "If value is 1/7 then ...". I do not want the algorithm to be able to focus on the survey data separately. (Of course it comes up with these reasons because >90% of the data comes from the second survey)

Is there a way to scale attributes so that this is not possible, even if this means a loss of information?

Edit: I guess a shift in feature distributions is inevitable (introducting bias). But I do not know how to keep that to a minimum.

$\endgroup$
1
$\begingroup$

Is there a way to scale attributes so that this is not possible, even if this means a loss of information?

Sure, recode the variable from the first survey to 1=1, 2=4, 3=7, or alternatively you could try something like 2=median from the second survey, but both choices are somehow arbitrary.

Notice however that you're making a very strong assumption in here that min category in first survey is equal to min in the second one, same with middle and max values. This assumption does not have to be true, since people may differ in styles of answering depending on how the questions are formatted. This may led to problems and should possibly be somehow controlled, e.g. by including a dummy for a survey type (if it's significant, you may have a problem).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.