Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
35
votes
2answers
15k views

how to weight KLD loss vs reconstruction loss in variational auto-encoder

in nearly all code examples I've seen of a VAE, the loss functions are defined as follows (this is tensorflow code, but I've seen similar for theano, torch etc. It's also for a convnet, but that's ...
35
votes
5answers
16k views

Free data set for very high dimensional classification [closed]

What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)? There is already a community wiki about free data sets: Locating ...
34
votes
8answers
2k views

Intuitive explanation of “Statistical Inference”

What is the cleanest, easiest way to explain someone the concept of Inference? What does it intuitively mean? How would you go to explain it to the layperson, or to a person who has studied a very ...
34
votes
4answers
38k views

What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
34
votes
3answers
4k views

Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

I have seen people have put a lot of efforts on SVM and Kernels, and they look pretty interesting as a starter in Machine Learning. But if we expect that almost-always we could find outperforming ...
34
votes
3answers
88k views

Polynomial regression using scikit-learn

I am trying to use scikit-learn for polynomial regression. From what I read polynomial regression is a special case of linear regression. I was hopping that maybe one of scikit's generalized linear ...
34
votes
4answers
42k views

How do you Interpret RMSLE (Root Mean Squared Logarithmic Error)?

I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem ...
34
votes
5answers
20k views

Can SVM do stream learning one example at a time?

I have a streaming data set, examples are available one at a time. I would need to do multi class classification on them. As soon as I fed a training example to the learning process, I have to discard ...
34
votes
1answer
9k views

Is there any algorithm combining classification and regression?

I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
34
votes
3answers
38k views

How to determine the quality of a multiclass classifier

Given a dataset with instances $x_i$ together with $N$ classes where every instance $x_i$ belongs exactly to one class $y_i$ a multiclass classifier After the training and testing I basically have a ...
34
votes
2answers
15k views

What does the term saturating nonlinearities mean?

I was reading the paper ImageNet Classification with Deep Convolutional Neural Networks and in section 3 were they explain the architecture of their Convolutional Neural Network they explain how they ...
34
votes
4answers
28k views

How do you use the 'test' dataset after cross-validation?

In some lectures and tutorials I've seen, they suggest to split your data into three parts: training, validation and test. But it is not clear how the test dataset should be used, nor how this ...
34
votes
3answers
24k views

Variable importance from SVM

How to obtain a variable (attribute) importance using SVM?
34
votes
5answers
74k views

How to determine the optimal threshold for a classifier and generate ROC curve?

Let say we have a SVM classifier, how do we generate ROC curve? (Like theoretically) (because we are generate TPR and FPR with each of the threshold). And how do we determine the optimal threshold for ...
33
votes
5answers
7k views

Why use regularisation in polynomial regression instead of lowering the degree?

When doing regression, for example, two hyper parameters to choose are often the capacity of the function (eg. the largest exponent of a polynomial), and the amount of regularisation. What I'm ...
33
votes
6answers
5k views

Data mining: How should I go about finding the functional form?

I'm curious about repeatable procedures that can be used to discover the functional form of the function y = f(A, B, C) + error_term where my only input is a set of ...
33
votes
6answers
20k views

How to get started with neural networks

I'm completely new to neural networks but highly interested in understanding them. However it's not easy at all to get started. Could anyone recommend a good book or any other kind of resource? Is ...
33
votes
2answers
12k views

What's the difference between “deep learning” and multilevel/hierarchical modeling?

Is "deep learning" just another term for multilevel/hierarchical modeling? I'm much more familiar with the latter than the former, but from what I can tell, the primary difference is not in their ...
33
votes
1answer
56k views

Training loss goes down and up again. What is happening?

My training loss goes down and then up again. It is very weird. The cross-validation loss tracks the training loss. What is going on? I have two stacked LSTMS as follows (on Keras): ...
33
votes
1answer
1k views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
32
votes
2answers
20k views

Cost function in OLS linear regression

I'm a bit confused with a lecture on linear regression given by Andrew Ng on Coursera about machine learning. There, he gave a cost function that minimises the sum-of-squares as: $$ \frac{1}{2m} \sum ...
32
votes
4answers
30k views

Implementation of CRF in python

Is there a popular implementation of Conditional Random Fields in Python? I can't seem to find any that is widely used and popular!
32
votes
3answers
23k views

Is whitening always good?

A common pre-processing step for machine learning algorithms is whitening of data. It seems like it is always good to do whitening since it de-correlates the data, making it simpler to model. When ...
32
votes
1answer
50k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
32
votes
6answers
25k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
32
votes
5answers
13k views

Can deep neural network approximate multiplication function without normalization?

Let say we want to do regression for simple f = x * y using standart deep neural network. I remember that there are reseraches that tells that NN with one hiden ...
32
votes
3answers
48k views

Why are bias nodes used in neural networks?

Why are bias nodes used in neural networks? How many you should use? In which layers you should use them: all hidden layers and the output layer?
32
votes
5answers
11k views

How to deal with hierarchical / nested data in machine learning

I'll explain my problem with an example. Suppose you want to predict the income of an individual given some attributes: {Age, Gender, Country, Region, City}. You have a training dataset like so <...
32
votes
3answers
27k views

Difference between a SVM and a perceptron

I am a bit confused with the difference between an SVM and a perceptron. Let me try to summarize my understanding here, and please feel free to correct where I am wrong and fill in what I have missed. ...
32
votes
4answers
14k views

Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
32
votes
4answers
17k views

Machine learning techniques for parsing strings?

I have a lot of address strings: 1600 Pennsylvania Ave, Washington, DC 20500 USA I want to parse them into their components: ...
31
votes
5answers
36k views

Neural network with skip-layer connections

I am interested in regression with neural networks. Neural networks with zero hidden nodes + skip-layer connections are linear models. What about the same neural nets but with hidden nodes ? I am ...
31
votes
3answers
7k views

Cross-validation including training, validation, and testing. Why do we need three subsets?

I have a question regarding the Cross-validation process. I am in the middle of a course of the Machine Learning on the Cursera. One of the topic is about the Cross-validation. I found it slightly ...
31
votes
8answers
83k views

In Naive Bayes, why bother with Laplace smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
31
votes
3answers
31k views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
31
votes
7answers
20k views

Inference vs. estimation?

What are the differences between "inference" and "estimation" under the context of machine learning? As a newbie, I feel that we infer random variables and estimate the model parameters. Is my this ...
31
votes
5answers
24k views

Should you ever standardise binary variables?

I have a data set with a set of features. Some of them are binary $(1=$ active or fired, $0=$ inactive or dormant), and the rest are real valued, e.g. $4564.342$. I want to feed this data to a ...
31
votes
6answers
25k views

Difference between Bayes network, neural network, decision tree and Petri nets

What is the difference between neural network, Bayesian network, decision tree and Petri nets, even though they are all graphical models and visually depict cause-effect relationship.
31
votes
2answers
76k views

libsvm data format [closed]

I'm using the libsvm (http://www.csie.ntu.edu.tw/~cjlin/libsvm/) tool for support vector classification. However, I'm confused about the format of the input data. From the README: The format of ...
31
votes
2answers
27k views

How to statistically compare the performance of machine learning classifiers?

Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing ...
31
votes
5answers
18k views

What does interaction depth mean in GBM?

I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in ...
31
votes
2answers
1k views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter) ...
30
votes
1answer
22k views

Explanation of min_child_weight in xgboost algorithm

The definition of the min_child_weight parameter in xgboost is given as the: minimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the ...
30
votes
3answers
3k views

Utility of feature-engineering : Why create new features based on existing features?

I often see people create new features based on existing features on a machine learning problem. For example, here : https://triangleinequality.wordpress.com/2013/09/08/basic-feature-engineering-with-...
30
votes
6answers
14k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
30
votes
3answers
8k views

Best bandit algorithm?

The most well-known bandit algorithm is upper confidence bound (UCB) which popularized this class of algorithms. Since then I presume there are now better algorithms. What is the current best ...
30
votes
3answers
40k views

How to judge if a supervised machine learning model is overfitting or not?

Can anyone tell me how to judge if a supervised machine learning model is overfitting or not? If I don't have an external validation dataset, I want to know if I can use ROC of 10 fold cross ...
30
votes
1answer
6k views

Why is PCA sensitive to outliers?

There are many posts on this SE that discuss robust approaches to principal component analysis (PCA), but I cannot find a single good explanation of why PCA is sensitive to outliers in the first place....
30
votes
3answers
11k views

In boosting, why are the learners “weak”?

See also a similar question on stats.SE. In boosting algorithms such as AdaBoost and LPBoost it is known that the "weak" learners to be combined only have to perform better than chance to be useful, ...
30
votes
2answers
35k views

In caret what is the real difference between cv and repeatedcv?

This is similar to question Caret re-sampling methods, although that really never answered this part of the question in an agreed upon way. caret's train function offers ...