2
$\begingroup$

We have an input composed of several features. Why is it better to use many of them in a Machine Learning algorithm, instead of using only the better discriminant one? And in which way many features are combined in order to produce a good model?

$\endgroup$
2
  • 4
    $\begingroup$ As long as the feature set includes variables that make sense as potential predictors of the output, the best choice among models admitting more than one feature will (nearly always) outperform a model based on the best single feature. It's not a certainty; it's possible to identify real examples where it won't happen but they're not the kind of problem that such techniques tend to be applied to. You should probably separate your two questions, though I believe the second is already addressed to some extent in a number of posts on site. $\endgroup$
    – Glen_b
    Jun 24 '18 at 9:57
  • 4
    $\begingroup$ Presumably you have taken exams before. Should the results (degree etc) be based on a single exam or multiple exams. $\endgroup$
    – seanv507
    Jun 24 '18 at 22:27
0
$\begingroup$

In general, you should add features carefully and try to add features which are not correlated with each other. Without given data, no one can predict if the model will be really good on just one feature. We definitely don't want to overfit but that doesn't mean we underfit and don't use the features which provide us discriminatory information.

What do you mean by "And in which way many features are combined in order to produce a good model ?" Are you asking a feature selection method?

$\endgroup$
0
$\begingroup$

This is not true in general.

Imagine that there’s a causal relationship between two events. In such a case, knowing that the cause has happened would be enough to predict the effect. A single feature would be enough.

Now imagine the opposite case, where you’re trying to predict a completely random, independent of everything, event. No matter how many features do you have, you wouldn’t be able to predict it. With more features you could only be more prone to overfit to the training data, gaining a false sense that you’re able to predict something.

Let's consider one more scenario, where you’re making weather forecasts. Having one data from a single weather station wouldn’t let you build a precise model. On another hand, having data from multiple weather stations gives you much more precise information on the weather conditions, helping you to build a much better water model.

You need accurate data, not a lot of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.