0
$\begingroup$

In using the neuralnet package, the input data set to the neuralnet call cannot have string/character variables - for example column "username" with say 10-20 different values, depending on the data set. To convert that I would prefer to be able to enumerate that column - so say "John" becomes 1, "Mike" becomes 2, etc, then reverse it when done with the model run, in case that is needed. However calls that I found so far - like model.matrix() or class.ind() will generate multiple columns - so if I got 5 distinct user names in the user column, it will generate five columns with 1/0 in each; That variability in number of columns creates the problem with the downstream code - is there a function that would enumerate that string column and return the map - then perhaps reverse call to remap back when all done?

$\endgroup$

migrated from stackoverflow.com Oct 5 '16 at 10:41

This question came from our site for professional and enthusiast programmers.

  • 2
    $\begingroup$ No code, not data. Must be stats methods question. Suggest migration. $\endgroup$ – DWin Oct 5 '16 at 0:37
0
$\begingroup$

Without details of your app, any advice is going to involve a lot of guesswork. All I can suggest is that you clearly identify usernames in the model: maybe a prefix like "uname_" and search for that in downstream code.

As for why an integer encoding is filled with modelling traps:

If "John" is 1 and "Mike" is 2, then all sorts of things are mathematically true:

John < Mike

Mike = John + 1

Mike = John + John

Mike = John * 2

etc.

Usually, the real-world interpretation of those equations makes no sense with categorical variables like names.

Even cases where there's an obvious order, like multiple-choice survey questions such as "strongly disagree / disagree / agree / strongly agree", an encoding of 1,2,3,4 makes no mathematical sense: four "strongly disagrees" do not make a "strongly agree".

This is why your model matrix has one column per name[1], and uses binary values to indicate the presence of the name in that data row. The names aren't numbers that can be added up, and the model structure has to reflect this to be useful.

Hope that helps.

[1] Or possibly one less column than the number of names - depends on the model.

$\endgroup$
  • $\begingroup$ Thank you Jason - yes that makes a lot of sense; However it still does not help me resolve the question of how do I deal with the variable number of 1/0 columns as expanded by matrix call - since I do not know the number of different user names in the user_name column in the example above? Granted generic code could be developed that deals with that but it still feels like something is missing. $\endgroup$ – Zoran Krunic Oct 5 '16 at 2:53
  • $\begingroup$ If you can post a "toy" example of some data using dput(), and some code highlighting your problem, it'll be possible to help you further. Without that, we can only hand-wave. :) $\endgroup$ – Jason Oct 5 '16 at 2:59
  • $\begingroup$ .. the more detailed note on the example - so I have operational data for servers - for each time snapshot there is one record - with columns like time, memory_used, cpu_used, program_name, user_name, etc ... with different levels of granularity - but besides number columns (memory, cpu) there are string columns - program_name, user_name for the user running the program, and those could be used to predict server performance and behavior patterns. That is where the user_name and program_name come in as string columns, with variable number of different values, generally unknown up-front. $\endgroup$ – Zoran Krunic Oct 5 '16 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy