Questions tagged [machine-learning]
Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.
2,146
questions
225
votes
10
answers
118k
views
Why is accuracy not the best measure for assessing classification models?
This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference.
...
Tim♦
- 127k
82
votes
4
answers
54k
views
Reduce Classification Probability Threshold
I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
- 1,369
271
votes
3
answers
28k
views
How to know that your machine learning problem is hopeless?
Imagine a standard machine-learning scenario:
You are confronted with a large multivariate dataset and you have a
pretty blurry understanding of it. What you need to do is to make
predictions ...
Tim♦
- 127k
98
votes
8
answers
35k
views
When is unbalanced data really a problem in Machine Learning?
We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
Tim♦
- 127k
24
votes
2
answers
6k
views
Proper scoring rule when there is a decision to make (e.g. spam vs ham email)
Among others on here, Frank Harrell is adamant about using proper scoring rules to assess classifiers. This makes sense. If we have 500 $0$s with $P(1)\in[0.45, 0.49]$ and 500 $1$s with $P(1)\in[0.51, ...
- 46.4k
116
votes
4
answers
45k
views
Why isn't Logistic Regression called Logistic Classification?
Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
- 2,136
555
votes
11
answers
639k
views
What is the difference between test set and validation set?
I found this confusing when I use the neural network toolbox in Matlab.
It divided the raw data set into three parts:
training set
validation set
test set
I notice in many training or learning ...
- 7,079
440
votes
5
answers
169k
views
How to understand the drawbacks of K-means
K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
- 6,647
191
votes
5
answers
69k
views
Training on the full dataset after cross-validation?
TL:DR: Is it ever a good idea to train an ML model on all the data available before shipping it to production? Put another way, is it ever ok to train on all data available and not check if the model ...
- 18.5k
486
votes
20
answers
172k
views
The Two Cultures: statistics vs. machine learning?
Last year, I read a blog post from Brendan O'Connor entitled "Statistics vs. Machine Learning, fight!" that discussed some of the differences between the two fields. Andrew Gelman responded favorably ...
57
votes
4
answers
26k
views
Can a random forest be used for feature selection in multiple linear regression?
Since RF can handle non-linearity but can't provide coefficients, would it be wise to use random forest to gather the most important features and then plug those features into a multiple linear ...
130
votes
10
answers
78k
views
Bias and variance in leave-one-out vs K-fold cross validation
How do different cross-validation methods compare in terms of model variance and bias?
My question is partly motivated by this thread: Optimal number of folds in $K$-fold cross-validation: is leave-...
- 18.5k
179
votes
4
answers
169k
views
Choice of K in K-fold cross-validation
I've been using the $K$-fold cross-validation a few times now to evaluate performance of some learning algorithms, but I've always been puzzled as to how I should choose the value of $K$.
I've often ...
- 2,337
345
votes
8
answers
141k
views
Why is Euclidean distance not a good metric in high dimensions?
I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
- 3,667
23
votes
2
answers
6k
views
What is happening here, when I use squared loss in logistic regression setting?
I am trying to use squared loss to do binary classification on a toy data set.
I am using mtcars data set, use mile per gallon and weight to predict transmission ...
- 34.7k
41
votes
2
answers
38k
views
Dropping one of the columns when using one-hot encoding
My understanding is that in machine learning it can be a problem if your dataset has highly correlated features, as they effectively encode the same information.
Recently someone pointed out that ...
- 718
62
votes
17
answers
12k
views
Machine learning cookbook / reference card / cheatsheet?
I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my ...
54
votes
4
answers
28k
views
Class imbalance in Supervised Machine Learning
This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
- 1,556
74
votes
12
answers
105k
views
Hold-out validation vs. cross-validation
To me, it seems that hold-out validation is useless. That is, splitting the original dataset into two-parts (training and testing) and using the testing score as a generalization measure, is somewhat ...
user46925
9
votes
2
answers
524
views
How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?
TLDR: What motivates the definition of $R^2$ in the Python function sklearn.metrics.r2_score?
DETAILS
The Python machine learning package ...
- 46.4k
50
votes
3
answers
51k
views
PCA and the train/test split
I have a dataset for which I have multiple sets of binary labels. For each set of labels, I train a classifier, evaluating it by cross-validation. I want to reduce dimensionality using principal ...
- 6,559
140
votes
9
answers
58k
views
Obtaining knowledge from a random forest
Random forests are considered to be black boxes, but recently I was thinking what knowledge can be obtained from a random forest?
The most obvious thing is the importance of the variables, in the ...
- 3,944
50
votes
1
answer
67k
views
How to interpret error measures?
I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. ...
- 603
102
votes
6
answers
44k
views
On the importance of the i.i.d. assumption in statistical learning
In statistical learning, implicitly or explicitly, one always assumes that the training set $\mathcal{D} = \{ \bf {X}, \bf{y} \}$ is composed of $N$ input/response tuples $({\bf{X}}_i,y_i)$ that are ...
- 1,426
92
votes
6
answers
48k
views
Feature selection for "final" model when performing cross-validation in machine learning
I am getting a bit confused about feature selection and machine learning
and I was wondering if you could help me out. I have a microarray dataset that is
classified into two groups and has 1000s of ...
- 2,425
32
votes
2
answers
13k
views
When should we discretize/bin continuous independent variables/features and when should not?
When should we discretize/bin independent variables/features and when should not?
My attempts to answer the question:
In general, we should not bin, because binning will lose information.
Binning is ...
- 34.7k
36
votes
3
answers
43k
views
Why is AUC higher for a classifier that is less accurate than for one that is more accurate?
I have two classifiers
A: naive Bayesian network
B: tree (singly-connected) Bayesian network
In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
- 1,337
133
votes
5
answers
77k
views
How does a Support Vector Machine (SVM) work?
How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? *
(* I'm ...
- 7,499
38
votes
2
answers
15k
views
Is there any algorithm combining classification and regression?
I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
- 561
18
votes
4
answers
12k
views
Are there any non-distance based clustering algorithms?
It seems that for K-means and other related algorithms, clustering is based off calculating distance between points. Is there one that works without it?
- 568
91
votes
3
answers
31k
views
Why is ridge regression called "ridge", why is it needed, and what happens when $\lambda$ goes to infinity?
Ridge regression coefficient estimate $\hat{\beta}^R$ are the values that minimize the
$$ \text{RSS} + \lambda \sum_{j=1}^p\beta_j^2. $$
My questions are:
If $\lambda = 0$, then we see that the ...
- 8,457
40
votes
3
answers
15k
views
Do we need gradient descent to find the coefficients of a linear regression model?
I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
- 6,235
34
votes
5
answers
13k
views
What problem does oversampling, undersampling, and SMOTE solve?
In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...
- 34.7k
60
votes
6
answers
35k
views
Practical hyperparameter optimization: Random vs. grid search
I'm currently going through Bengio's and Bergstra's Random Search for Hyper-Parameter Optimization [1] where the authors claim random search is more efficient than grid search in achieving ...
- 2,632
22
votes
2
answers
8k
views
How does linear discriminant analysis reduce the dimensions?
There are words from "The Elements of Statistical Learning" on page 91:
The K centroids in p-dimensional input space span at most K-1 dimensional subspace, and if p
is much larger than K, this ...
- 435
14
votes
1
answer
11k
views
How to know if a learning curve from SVM model suffers from bias or variance?
I created this learning curve and I want to know if my SVM model suffers from bias or variance? How can I conclude that from this graph?
- 277
119
votes
7
answers
85k
views
Why use gradient descent for linear regression, when a closed-form math solution is available?
I am taking the Machine Learning courses online and learnt about Gradient Descent for calculating the optimal values in the hypothesis.
h(x) = B0 + B1X
why we ...
- 1,293
118
votes
6
answers
50k
views
Is it possible to train a neural network without backpropagation?
Many neural network books and tutorials spend a lot of time on the backpropagation algorithm, which is essentially a tool to compute the gradient.
Let's assume we are building a model with ~10K ...
- 34.7k
64
votes
7
answers
48k
views
Binary classification with strongly unbalanced classes
I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
- 842
62
votes
7
answers
134k
views
Is it a good practice to always scale/normalize data for machine learning? [duplicate]
My understanding is that when some features have different ranges in their values (for example, imagine one feature being the age of a person and another one being their salary in USD) will affect ...
24
votes
3
answers
12k
views
Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity
I have a test set of 100 cases and two classifiers.
I generated predictions and computed ROC AUC, sensitivity and specificity for both classifiers.
Question 1: How can I compute p-value to check if ...
- 255
244
votes
9
answers
116k
views
Why is Newton's method not widely used in machine learning?
This is something that has been bugging me for a while, and I couldn't find any satisfactory answers online, so here goes:
After reviewing a set of lectures on convex optimization, Newton's method ...
- 2,431
192
votes
10
answers
45k
views
Why the sudden fascination with tensors?
I've noticed lately that a lot of people are developing tensor equivalents of many methods (tensor factorization, tensor kernels, tensors for topic modeling, etc) I'm wondering, why is the world ...
124
votes
4
answers
240k
views
Softmax vs Sigmoid function in Logistic classifier?
What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ?
Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
- 1,725
80
votes
8
answers
118k
views
How and why do normalization and feature scaling work?
I see that lots of machine learning algorithms work better with mean cancellation and covariance equalization. For example, Neural Networks tend to converge faster, and K-Means generally gives better ...
- 1,667
76
votes
6
answers
47k
views
Why is multicollinearity not checked in modern statistics/machine learning
In traditional statistics, while building a model, we check for multicollinearity using methods such as estimates of the variance inflation factor (VIF), but in machine learning, we instead use ...
- 761
19
votes
3
answers
3k
views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
- 694
18
votes
2
answers
3k
views
Example when using accuracy as an outcome measure will lead to a wrong conclusion
I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
- 7,053
247
votes
4
answers
116k
views
ROC vs precision-and-recall curves
I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other.
Do they always provide complementary insight about the performance of a ...
- 18.5k
37
votes
3
answers
56k
views
What algorithms need feature scaling, beside from SVM?
I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
- 1,139