3
$\begingroup$

I know that age is often represented as a numerical feature. For example in linear regression, it is common to use one single independent variable (IV) to represent it. However, one single IV cannot capture non-linear relationship. In some other models, such as factorization machines, it is advised to use dummy variable in order to capture feature interaction.

So what is the good way to represent ordinal/ratio data in neural network, especially deep learning models? Numerical feature? Dummy variable?

Improve this question
$\endgroup$
1
  • $\begingroup$ It's not correct that a single independent variable (IV is usually used to refer to instrumental variables) cannot capture non-linear relationship in neural networks. That is true in linear models, because there is only one coefficient to express the relationship. A neural network with multiple nodes (and multiple layers for complex relationships) captures non-linearities. $\endgroup$
    – Johannes
    Commented Mar 15, 2019 at 9:54

2 Answers 2

Reset to default
1
$\begingroup$

Variables cannot "capture non-linear relationship", because the relationship that you are talking about is a function that we are trying to learn, not-linearity is about the function, not the inputs to this function. Linear regression is able to learn a function that is linear in parameters, while more complicated machine learning algorithms can be able to learn more complicated functions.

Moreover, you can think of regression as of a very simplified special case of neural network, so if something works for regression, it should work for neural networks as well (i.e. treat them as numerical feature). Creating features based on those variables would be feature engineering and we usually use neural networks because they don't need feature engineering and learn the features by themselves.

Finally, using dummy variables for things like age is a bad idea. With using dummy variables, your algorithm needs to find individual rules for each of the levels of the dummy variable, so it finds separate rules for people who are 30 years old, and for the ones who are 31 years old. This is not the case with age, where you don't assume that people of any possible age could behave completely different then others.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ "Finally, using dummy variables for things like age is a bad idea." – One thing you can do is use dummy variables for "age is at least X", so that someone who is 30 would have the first 30 variables set to 1 and the remaining 70 (or however old you go) set to 0. This could ideally make the representation a little easier, though it's not without its own weirdness. Ideally in a deep network this wouldn't be necessary, but I've seen it be useful there as well. $\endgroup$
    – Danica
    Commented Mar 15, 2019 at 12:15
0
$\begingroup$

I know for time, a common way to represent the variable is using a sinusoidal function. This allows, for example, 2PM on Monday to be the same value as 2PM on Friday.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.