Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
48
votes
1answer
20k views

Understanding “almost all local minimum have very similar function value to the global optimum”

In a recent blog post by Rong Ge, it was said that: It is believed that for many problems including learning deep nets, almost all local minimum have very similar function value to the global ...
48
votes
4answers
24k views

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
48
votes
7answers
4k views

Where to start with statistics for an experienced developer

During the first half of 2015 I did the coursera course of Machine Learning (by Andrew Ng, GREAT course). And learned the basics of machine learning (linear regression, logistic regression, SVM, ...
48
votes
2answers
49k views

What is maxout in neural network?

Can anyone explain what maxout units in a neural network do? How do they perform and how do they differ from conventional units? I tried to read the 2013 "Maxout Network" paper by Goodfellow et al. (...
47
votes
1answer
27k views

What is an ablation study? And is there a systematic way to perform it?

What is an ablation study? And is there a systematic way to perform it? For example, I have $n$ predictors in a linear regression which I will call as my model. How will I perform an ablation study ...
47
votes
7answers
100k views

Why is the validation accuracy fluctuating?

I have a four layer CNN to predict response to cancer using MRI data. I use ReLU activations to introduce nonlinearities. The train accuracy and loss monotonically increase and decrease respectively. ...
46
votes
4answers
23k views

How are kernels applied to feature maps to produce other feature maps?

I am trying to understand the convolution part of convolutional neural networks. Looking at the following figure: I have no problems understanding the first convolution layer where we have 4 ...
45
votes
7answers
93k views

Data normalization and standardization in neural networks

I am trying to predict the outcome of a complex system using neural networks (ANN's). The outcome (dependent) values range between 0 and 10,000. The different input variables have different ranges. ...
45
votes
1answer
57k views

How to interpret error measures?

I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. ...
45
votes
3answers
42k views

What are the differences between 'epoch', 'batch', and 'minibatch'?

As far as I know, when adopting Stochastic Gradient Descent as learning algorithm, someone use 'epoch' for full dataset, and 'batch' for data used in a single update step, while another use 'batch' ...
44
votes
13answers
24k views

Can machine learning decode the SHA256 hashes?

I have a 64 character SHA256 hash. I'm hoping to train a model that can predict if the plaintext used to generate the hash begins with a 1 or not. Regardless if this is "Possible", what algorithm ...
43
votes
5answers
34k views

How does rectilinear activation function solve the vanishing gradient problem in neural networks?

I found rectified linear unit (ReLU) praised at several places as a solution to the vanishing gradient problem for neural networks. That is, one uses max(0,x) as activation function. When the ...
43
votes
5answers
9k views

Is machine learning less useful for understanding causality, thus less interesting for social science?

My understanding of the difference between machine learning/other statistical predictive techniques vs. the kind of statistics that social scientists (e.g., economists) use is that economists seem ...
43
votes
1answer
23k views

How is softmax_cross_entropy_with_logits different from softmax_cross_entropy_with_logits_v2?

Specifically, I suppose I wonder about this statement: Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default. Which is shown when I ...
43
votes
2answers
59k views

Measures of variable importance in random forests

I've been playing around with random forests for regression and am having difficulty working out exactly what the two measures of importance mean, and how they should be interpreted. The ...
43
votes
3answers
43k views

Online vs offline learning?

What is the difference between offline and online learning? Is it just a matter of learning over the entire dataset (offline) vs. learning incrementally (one instance at a time)? What are examples ...
42
votes
5answers
31k views

Why is tanh almost always better than sigmoid as an activation function?

In Andrew Ng's Neural Networks and Deep Learning course on Coursera he says that using $tanh$ is almost always preferable to using $sigmoid$. The reason he gives is that the outputs using $tanh$ ...
41
votes
1answer
43k views

CNN architectures for regression?

I've been working on a regression problem where the input is an image, and the label is a continuous value between 80 and 350. The images are of some chemicals after a reaction takes place. The color ...
41
votes
3answers
86k views

How to interpret Mean Decrease in Accuracy and Mean Decrease GINI in Random Forest models

I'm having some difficulty understanding how to interpret variable importance output from the Random Forest package. Mean decrease in accuracy is usually described as "the decrease in model accuracy ...
41
votes
5answers
19k views

LDA vs word2vec

I am trying to understand what is similarity between Latent Dirichlet Allocation and word2vec for calculating word similarity. As I understand, LDA maps words to a vector of probabilities of latent ...
41
votes
1answer
8k views

Variational inference versus MCMC: when to choose one over the other?

I think I get the general idea of both VI and MCMC including the various flavors of MCMC like Gibbs sampling, Metropolis Hastings etc. This paper provides a wonderful exposition of both methods. I ...
40
votes
4answers
76k views

Recall and precision in classification

I read some definitions of recall and precision, though it is every time in the context of information retrieval. I was wondering if someone could explain this a bit more in a classification context ...
40
votes
5answers
32k views

Cost function of neural network is non-convex?

The cost function of neural network is $J(W,b)$, and it is claimed to be non-convex. I don't quite understand why it's that way, since as I see that it's quite similar to the cost function of logistic ...
40
votes
2answers
6k views

If only prediction is of interest, why use lasso over ridge?

On page 223 in An Introduction to Statistical Learning, the authors summarise the differences between ridge regression and lasso. They provide an example (Figure 6.9) of when "lasso tends to ...
40
votes
3answers
26k views

PCA and the train/test split

I have a dataset for which I have multiple sets of binary labels. For each set of labels, I train a classifier, evaluating it by cross-validation. I want to reduce dimensionality using principal ...
40
votes
6answers
72k views

Improve classification with many categorical variables

I'm working on a dataset with 200,000+ samples and approximately 50 features per sample: 10 continuous variables and the other ~40 are categorical variables (countries, languages, scientific fields ...
39
votes
6answers
23k views

Why do I get a 100% accuracy decision tree?

I'm getting a 100% accuracy for my decision tree. What am I doing wrong? This is my code: ...
39
votes
3answers
21k views

Intuitive difference between hidden Markov models and conditional random fields

I understand that HMMs (Hidden Markov Models) are generative models, and CRF are discriminative models. I also understand how CRFs (Conditional Random Fields) are designed and used. What I do not ...
38
votes
4answers
27k views

Is a strong background in maths a total requisite for ML?

I'm starting to want to advance my own skillset and I've always been fascinated by machine learning. However, six years ago instead of pursuing this I decided to take a completely unrelated degree to ...
38
votes
4answers
30k views

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? ...
38
votes
3answers
31k views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
38
votes
3answers
1k views

Application of machine learning methods in StackExchange websites

I have a Machine Learning course this semester and the professor asked us to find a real-world problem and solve it by one of machine learning methods introduced in the class, as: Decision Trees ...
38
votes
3answers
22k views

Creating a “certainty score” from the votes in random forests?

I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set ...
38
votes
3answers
4k views

Variance of $K$-fold cross-validation estimates as $f(K)$: what is the role of “stability”?

TL,DR: It appears that, contrary to oft-repeated advice, leave-one-out cross validation (LOO-CV) -- that is, $K$-fold CV with $K$ (the number of folds) equal to $N$ (the number of training ...
37
votes
4answers
31k views

When should I balance classes in a training data set?

I had an online course, where I learned, that unbalanced classes in the training data might lead to problems, because classification algorithms go for the majority rule, as it gives good results if ...
37
votes
5answers
40k views

What exactly is a Bayesian model?

Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad. So what exactly is a Bayesian model?
37
votes
2answers
16k views

Gradient Boosting for Linear Regression - why does it not work?

While learning about Gradient Boosting, I haven't heard about any constraints regarding the properties of a "weak classifier" that the method uses to build and ensemble model. However, I could not ...
37
votes
1answer
65k views

what does the numbers in the classification report of sklearn mean?

I have below an example I pulled from sklearn 's sklearn.metrics.classification_report documentation. What I don't understand is why there are f1-score, precision and recall values for each class ...
37
votes
4answers
4k views

Cloud computing platforms for machine learning [closed]

I've got a small list of companies that provide a platform for running R, python, or octave scripts on clusters built on top of amazon EC2. Are there other names I should add? Cloudnumbers Opani ...
36
votes
4answers
5k views

Is LSTM (Long Short-Term Memory) dead?

From my own experience, LSTM has a long training time, and does not improve performance significantly in many real world tasks. To make the question more specific, I want to ask when LSTM will work ...
36
votes
1answer
33k views

Relative variable importance for Boosting

I'm looking for an explanation of how relative variable importance is computed in Gradient Boosted Trees that is not overly general/simplistic like: The measures are based on the number of times a ...
36
votes
6answers
39k views

How to choose between ROC AUC and F1 score?

I recently completed a Kaggle competition in which roc auc score was used as per competition requirement. Before this project, I normally used f1 score as the metric to measure model performance. ...
36
votes
2answers
10k views

Do we need gradient descent to find the coefficients of a linear regression model?

I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
36
votes
2answers
9k views

Why is mean squared error the cross-entropy between the empirical distribution and a Gaussian model?

In 5.5, Deep Learning (by Ian Goodfellow, Yoshua Bengio and Aaron Courville), it states that Any loss consisting of a negative log-likelihood is a cross-entropy between the empirical distribution ...
36
votes
3answers
43k views

Things to consider about masters programs in statistics

It is admission season for graduate schools. I (and many students like me) am now trying to decide which statistics program to pick. What are some things those of you who work with statistics suggest ...
36
votes
2answers
36k views

Neural Network: For Binary Classification use 1 or 2 output neurons?

Assume I want to do binary classification (something belongs to class A or class B). There are some possibilities to do this in the output layer of a neural network: Use 1 output node. Output 0 (<...
35
votes
3answers
16k views

Pre-training in deep convolutional neural network?

Have anyone seen any literature on pre-training in deep convolutional neural network? I have only seen unsupervised pre-training in autoencoder or restricted boltzman machines.
35
votes
5answers
5k views

How to handle a “self defeating” prediction model?

I was watching a presentation by an ML specialist from a major retailer, where they had developed a model to predict out of stock events. Let's assume for a moment that over time, their model ...
35
votes
2answers
45k views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
35
votes
5answers
4k views

Can you overfit by training machine learning algorithms using CV/Bootstrap?

This question may well be too open-ended to get a definitive answer, but hopefully not. Machine learning algorithms, such as SVM, GBM, Random Forest etc, generally have some free parameters that, ...