I would like to create a custom metric that takes into account multiple features and produces a certain score.

However, the issue is that I have data that has many different data types, such as nominal, categorical, continuous and integers. Plus the continuous features lay on different scales.

1) How could I include all these different data types in a new metric?

2) How to solve the scale problem? I was thinking either centering and scaling continuous features or using z score on all of the features.

3) Would this be more of a composite measure, weighted average or something else?


I'm posting as an answer because the comment became too long. :)

Usually, this type of approach is better after some sort of standardization of the variables to make them comparable. So, we may transform the variables so that they make sense in similar ranges. The advantage of having the variables living more or less in the same range is that the coefficients of a weighted average would represent the relative importance of each variable.

Continuous variables can be transformed, as you mentioned, on their z-score, i.e., by $z = \frac{x-\mu}{\sigma}$, where $\mu$ and $\sigma$ are the mean and the standard deviation, respectively. The variable $z$ will have a mean of $0$ and a standard deviation of $1$. If you have several variables of this type, the transformed ones will be comparable.

Integers may be transformed in the same way if there are may different values or just divided by the maximum value if we have non-negative integers. Dividing by the maximum value will make the maximum of the transformed variable be equal to $1$.

Categorical variables (that include nominal) can be transformed into dummy variables. These assume the value $1$ if that category was present for that individual and $0$ otherwise.

This is just an initial approach and it may not be perfect. For instance, if the continuous variables or the integers are skewed you may want to use other transformations, say the logarithm of the data, or the square root or a Box-Cox transformation.

| cite | improve this answer | |
  • $\begingroup$ Thank you for the feedback. Your recommendation is more inclined towards data transformation for machine learning, however i am more looking in creating a feature that is an index of some sort. en.wikipedia.org/wiki/Scale_(social_sciences) $\endgroup$ – Loncar Apr 23 '19 at 13:10
  • $\begingroup$ In general we can have several variables available, but then we can use some selection mechanism to have a smaller subset that gives similar results. Creating an index will require some validation afterwards. Note that using an existing index has the advantage of the results being comparable with published articles. $\endgroup$ – Ertxiem - reinstate Monica Apr 23 '19 at 14:56
  • $\begingroup$ Could you provide some source, where I can read more about creating indexes or a custom metric, please? $\endgroup$ – Loncar Apr 24 '19 at 7:17
  • $\begingroup$ After the initial selection of variables, I've used the Cosmin checklist, there is also an article. I know that there are other tools like this one - the Cosmin checklist is more targetted to the health sciences. $\endgroup$ – Ertxiem - reinstate Monica Apr 24 '19 at 8:16
  • $\begingroup$ Thanks. What do you thing about this package, benchmarking. I think it could do the trick. rdrr.io/cran/Benchmarking/man/Benchmarking-package.html $\endgroup$ – Loncar Apr 24 '19 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.