I am newbie in machine learning. I have been studying about features extraction and some classification approaches, in the term of my study, I have a question in my mind, what the reasons we need to extract a lot of features for classification? is it possible to extract one reliable feature from a dataset for good classification results?
As ever, it depends on the dataset. Sometimes, one feature may be sufficient to build a highly-accurate classifier, but on most interesting, non-trivial problems, multiple features are needed. For example, in image processing one often needs a large number of weak features. Note that this is a separate issue from data collection, where you have some control over what to measure in the first place. Some expert knowledge at that early stage can save a lot of work on later feature selection/dimension reduction etc.
Collecting "good" features (e.g., such features that lead to maximum accuracy) in advance, is in general, not always possible. Therefore, we collect "alot" of features in the hope that "good" features are among them.
After we collected alot of features we often use feature selection/extraction methods, in order to reduce them to a smaller set of features, that lead to promising results. However, this doesn't always work as there are many pitfalls. For example there are "bad" features that in combination can work much better as a single "good" feature. So we must take this also into account.
Here you can find 7 techniques for data dimensionality reduction. Have a look on these...
Yes it is possible getting great classification results using one feature.
Think of the null example - using the class itself as a feature for classification.
In general, the more complex are the interactions between your different target classes, the more feature you will need.