Skip to main content

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
243 votes
11 answers

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
Tim's user avatar
  • 139k
86 votes
4 answers

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
sdgaw erzswer's user avatar
289 votes
3 answers

How to know that your machine learning problem is hopeless?

Imagine a standard machine-learning scenario: You are confronted with a large multivariate dataset and you have a pretty blurry understanding of it. What you need to do is to make predictions ...
Tim's user avatar
  • 139k
103 votes
8 answers

When is unbalanced data really a problem in Machine Learning?

We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
Tim's user avatar
  • 139k
29 votes
2 answers

Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

Among others on here, Frank Harrell is adamant about using proper scoring rules to assess classifiers. This makes sense. If we have 500 $0$s with $P(1)\in[0.45, 0.49]$ and 500 $1$s with $P(1)\in[0.51, ...
Dave's user avatar
  • 63.3k
119 votes
4 answers

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
Ismael Ghalimi's user avatar
563 votes
11 answers

What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
xiaohan2012's user avatar
  • 7,179
13 votes
2 answers

How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?

TLDR: What motivates the definition of $R^2$ in the Python function sklearn.metrics.r2_score? DETAILS The Python machine learning package ...
Dave's user avatar
  • 63.3k
201 votes
6 answers

Training on the full dataset after cross-validation?

TL:DR: Is it ever a good idea to train an ML model on all the data available before shipping it to production? Put another way, is it ever ok to train on all data available and not check if the model ...
Amelio Vazquez-Reina's user avatar
444 votes
5 answers

How to understand the drawbacks of K-means

K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
KevinKim's user avatar
  • 6,899
12 votes
1 answer

Interpreting nonlinear regression $R^2$

In ordinary least squares linear regression, $R^2=1-\frac{SSRes}{SSTotal}$ is described as the “proportion of variance explained”. Does this apply to nonlinear regression, too?
Dave's user avatar
  • 63.3k
496 votes
20 answers

The Two Cultures: statistics vs. machine learning?

Last year, I read a blog post from Brendan O'Connor entitled "Statistics vs. Machine Learning, fight!" that discussed some of the differences between the two fields. Andrew Gelman responded favorably ...
60 votes
4 answers

Can a random forest be used for feature selection in multiple linear regression?

Since RF can handle non-linearity but can't provide coefficients, would it be wise to use random forest to gather the most important features and then plug those features into a multiple linear ...
Hidden Markov Model's user avatar
140 votes
10 answers

Bias and variance in leave-one-out vs K-fold cross validation

How do different cross-validation methods compare in terms of model variance and bias? My question is partly motivated by this thread: Optimal number of folds in $K$-fold cross-validation: is leave-...
Amelio Vazquez-Reina's user avatar
182 votes
4 answers

Choice of K in K-fold cross-validation

I've been using the $K$-fold cross-validation a few times now to evaluate performance of some learning algorithms, but I've always been puzzled as to how I should choose the value of $K$. I've often ...
Charles Menguy's user avatar
43 votes
2 answers

Dropping one of the columns when using one-hot encoding

My understanding is that in machine learning it can be a problem if your dataset has highly correlated features, as they effectively encode the same information. Recently someone pointed out that ...
dasboth's user avatar
  • 738
28 votes
2 answers

What is happening here, when I use squared loss in logistic regression setting?

I am trying to use squared loss to do binary classification on a toy data set. I am using mtcars data set, use mile per gallon and weight to predict transmission ...
Haitao Du's user avatar
  • 37k
356 votes
8 answers

Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
teaLeef's user avatar
  • 3,777
33 votes
2 answers

When should we discretize/bin continuous independent variables/features and when should not?

When should we discretize/bin independent variables/features and when should not? My attempts to answer the question: In general, we should not bin, because binning will lose information. Binning is ...
Haitao Du's user avatar
  • 37k
64 votes
17 answers

Machine learning cookbook / reference card / cheatsheet?

I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my ...
54 votes
4 answers

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
NG_21's user avatar
  • 1,556
75 votes
12 answers

Hold-out validation vs. cross-validation

To me, it seems that hold-out validation is useless. That is, splitting the original dataset into two-parts (training and testing) and using the testing score as a generalization measure, is somewhat ...
user avatar
50 votes
3 answers

PCA and the train/test split

I have a dataset for which I have multiple sets of binary labels. For each set of labels, I train a classifier, evaluating it by cross-validation. I want to reduce dimensionality using principal ...
Bitwise's user avatar
  • 6,629
141 votes
9 answers

Obtaining knowledge from a random forest

Random forests are considered to be black boxes, but recently I was thinking what knowledge can be obtained from a random forest? The most obvious thing is the importance of the variables, in the ...
Tomek Tarczynski's user avatar
7 votes
3 answers

$R^2$ on out-sample data set

The conventional definition of $R^2$ is: $R^2 = 1-SSE/SST$, where SSE denotes sum of squared errors and SST is total sum of squares ($n\times variance$, n being number of sample points in train set). ...
Maaz's user avatar
  • 317
2 votes
1 answer

Ridge regression for multicollinearity and outliers

I'm wondering about techniques like ridge regression with regard to both multicollinearity and outliers. My understanding is that ridge regression is primarily used for multicollinearity, but that ...
fmtcs's user avatar
  • 535
111 votes
6 answers

On the importance of the i.i.d. assumption in statistical learning

In statistical learning, implicitly or explicitly, one always assumes that the training set $\mathcal{D} = \{ \bf {X}, \bf{y} \}$ is composed of $N$ input/response tuples $({\bf{X}}_i,y_i)$ that are ...
Quantuple's user avatar
  • 1,546
95 votes
6 answers

Feature selection for "final" model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
danielsbrewer's user avatar
52 votes
1 answer

How to interpret error measures?

I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. ...
FloIancu's user avatar
  • 623
39 votes
2 answers

Is there any algorithm combining classification and regression?

I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
Shudong's user avatar
  • 571
4 votes
2 answers

Is the proportion classified correctly a reasonable analogue of $R^2$ for a classification model?

Let's do some classification and evaluate the prediction quality. The easiest metric to understand is the prediction accuracy, which can be reported as the proportion classified correctly to put the ...
Dave's user avatar
  • 63.3k
21 votes
4 answers

Why does regularization wreck orthogonality of predictions and residuals in linear regression?

Following up on this question... In ordinary least squares, the predictions and residuals are orthogonal. $$\sum_{i=1}^n\hat{y}_i (y_i - \hat{y}_i) = 0$$ If we estimate the regression coefficients ...
Dave's user avatar
  • 63.3k
134 votes
5 answers

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
tdc's user avatar
  • 7,559
40 votes
3 answers

Do we need gradient descent to find the coefficients of a linear regression model?

I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
Victor's user avatar
  • 6,595
37 votes
3 answers

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
Jane Wayne's user avatar
  • 1,399
35 votes
5 answers

What problem does oversampling, undersampling, and SMOTE solve?

In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...
Matthew Drury's user avatar
18 votes
4 answers

Are there any non-distance based clustering algorithms?

It seems that for K-means and other related algorithms, clustering is based off calculating distance between points. Is there one that works without it?
user154510's user avatar
94 votes
3 answers

Why is ridge regression called "ridge", why is it needed, and what happens when $\lambda$ goes to infinity?

Ridge regression coefficient estimate $\hat{\beta}^R$ are the values that minimize the $$ \text{RSS} + \lambda \sum_{j=1}^p\beta_j^2. $$ My questions are: If $\lambda = 0$, then we see that the ...
cgo's user avatar
  • 9,167
63 votes
6 answers

Practical hyperparameter optimization: Random vs. grid search

I'm currently going through Bengio's and Bergstra's Random Search for Hyper-Parameter Optimization [1] where the authors claim random search is more efficient than grid search in achieving ...
Bar's user avatar
  • 2,862
22 votes
2 answers

How does linear discriminant analysis reduce the dimensions?

There are words from "The Elements of Statistical Learning" on page 91: The K centroids in p-dimensional input space span at most K-1 dimensional subspace, and if p is much larger than K, this ...
jerry_sjtu's user avatar
126 votes
7 answers

Why use gradient descent for linear regression, when a closed-form math solution is available?

I am taking the Machine Learning courses online and learnt about Gradient Descent for calculating the optimal values in the hypothesis. h(x) = B0 + B1X why we ...
Purus's user avatar
  • 1,363
85 votes
6 answers

Variable selection for predictive modeling really needed in 2016?

This question has been asked on CV some yrs ago, it seems worth a repost in light of 1) order of magnitude better computing technology (e.g. parallel computing, HPC etc) and 2) newer techniques, e.g. [...
horaceT's user avatar
  • 3,352
78 votes
6 answers

Why is multicollinearity not checked in modern statistics/machine learning

In traditional statistics, while building a model, we check for multicollinearity using methods such as estimates of the variance inflation factor (VIF), but in machine learning, we instead use ...
user's user avatar
  • 781
20 votes
3 answers

Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?

The i.i.d. assumption states: We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion. To me, ...
Olórin's user avatar
  • 724
14 votes
1 answer

How to know if a learning curve from SVM model suffers from bias or variance?

I created this learning curve and I want to know if my SVM model suffers from bias or variance? How can I conclude that from this graph?
Afke's user avatar
  • 277
120 votes
6 answers

Is it possible to train a neural network without backpropagation?

Many neural network books and tutorials spend a lot of time on the backpropagation algorithm, which is essentially a tool to compute the gradient. Let's assume we are building a model with ~10K ...
Haitao Du's user avatar
  • 37k
66 votes
7 answers

Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
LazyCat's user avatar
  • 862
63 votes
7 answers

Is it a good practice to always scale/normalize data for machine learning? [duplicate]

My understanding is that when some features have different ranges in their values (for example, imagine one feature being the age of a person and another one being their salary in USD) will affect ...
Juan Antonio Gomez Moriano's user avatar
24 votes
3 answers

Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity

I have a test set of 100 cases and two classifiers. I generated predictions and computed ROC AUC, sensitivity and specificity for both classifiers. Question 1: How can I compute p-value to check if ...
kostek's user avatar
  • 255
18 votes
2 answers

Example when using accuracy as an outcome measure will lead to a wrong conclusion

I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
rep_ho's user avatar
  • 7,639

2 3 4 5