Machine learning method to determine continuous values from discrete and continuous parameters

Question

I watched online courses about multivariable linear regression which addresses the problem of determining values from numeric inputs. Like, predict prices for houses based on age, size, number of floors...

I also learned about things like logistic regression which concerns classification problems based on numerical inputs. Like, predict if an email is spam or not spam.

But I'm having trouble to find informations about predicting continuous values (like a price) based on a mix of discrete and continuous parameters.

For example, if I have several models of laptops classified with these categories:

[discrete] color: gray, blue, black...
[discrete] backlit_keyboard: yes, no
[discrete] material: plastic, aluminium
[continuous] weight: (numeric)
[discrete] brand: apple, sony, lenovo...

Now, assuming that the price will be a function of all these variables, and that I have a large training set of actual prices for each described object, I want to determine this function so that I'll be able to tell an estimated price for an object which has no previous example in the training set.

Which machine learning methods could/should be used to fit such a purpose?

Answer 1

One standard thing to do is to use one-hot encoding, and then run any regression algorithm you'd like (e.g. a variant of linear regression, or maybe kernel ridge regression). That is, the first dimension can be color_is_gray (which would be 1 if the color is gray, and 0 if not); the second color_is_blue, and so on. Then concatenate features for the other attributes.

If you have some notion of distance between the discrete attributes, you can instead perform multidimensional scaling and use the features it obtains.

Answer 2

Adding to what was already said: A simple approach to represent categorical features in a model (whatever model you use) is one-hot-encoding. This translates the categories to indicators and the representation is again numeric.

A rather novel idea is to apply feature embeddings. The idea is to go beyond simple indicator vectors of zeros and ones. Rather, the aim is to represent relations between the levels of the categorical feature. In this blog post Exploring Embeddings for Categorical Variables with Keras the 7 days of the week (+ a level for "missing") are embedded in a lower dimensional space with the result of improved generalization. The embedding uses the target (bike count data) and the position of e.g. Saturday is closer to Sunday (and close to Tuesday) than to Wednesday. It is debatable whether this is true but one-hot-encoding would not be able to achieve this.