We have an input composed of several features. Why is it better to use many of them in a Machine Learning algorithm, instead of using only the better discriminant one? And in which way many features are combined in order to produce a good model?
In general, you should add features carefully and try to add features which are not correlated with each other. Without given data, no one can predict if the model will be really good on just one feature. We definitely don't want to overfit but that doesn't mean we underfit and don't use the features which provide us discriminatory information.
What do you mean by "And in which way many features are combined in order to produce a good model ?" Are you asking a feature selection method?
This is not true in general.
Imagine that there’s a causal relationship between two events. In such a case, knowing that the cause has happened would be enough to predict the effect. A single feature would be enough.
Now imagine the opposite case, where you’re trying to predict a completely random, independent of everything, event. No matter how many features do you have, you wouldn’t be able to predict it. With more features you could only be more prone to overfit to the training data, gaining a false sense that you’re able to predict something.
Let's consider one more scenario, where you’re making weather forecasts. Having one data from a single weather station wouldn’t let you build a precise model. On another hand, having data from multiple weather stations gives you much more precise information on the weather conditions, helping you to build a much better water model.
You need accurate data, not a lot of data.