Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

103
votes
2answers
107k views

Gradient Boosting Tree vs Random Forest

Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there ...
83
votes
4answers
53k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
65
votes
9answers
72k views

What algorithm should I use to detect anomalies on time-series?

Background I'm working in Network Operations Center, we monitor computer systems and their performance. One of the key metrics to monitor is a number of visitors\customers currently connected to our ...
68
votes
7answers
24k views

Euclidean distance is usually not good for sparse data?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
32
votes
5answers
18k views

How to get started with neural networks

I'm completely new to neural networks but highly interested in understanding them. However it's not easy at all to get started. Could anyone recommend a good book or any other kind of resource? Is ...
52
votes
3answers
29k views

Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...
47
votes
4answers
37k views

Why not approach classification through regression?

Some material I've seen on machine learning said that it's a bad idea to approach a classification problem through regression. But I think it's always possible to do a continuous regression to fit the ...
37
votes
3answers
29k views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
40
votes
6answers
23k views

Why downsample?

Suppose I want to learn a classifier that predicts if an email is spam. And suppose only 1% of emails are spam. The easiest thing to do would be to learn the trivial classifier that says none of the ...
29
votes
7answers
13k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
32
votes
4answers
32k views

What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
24
votes
2answers
32k views

How to judge if a supervised machine learning model is overfitting or not?

Can anyone tell me how to judge if a supervised machine learning model is overfitting or not? If I don't have an external validation dataset, I want to know if I can use ROC of 10 fold cross ...
36
votes
3answers
42k views

Things to consider about masters programs in statistics

It is admission season for graduate schools. I (and many students like me) am now trying to decide which statistics program to pick. What are some things those of you who work with statistics suggest ...
35
votes
3answers
21k views

Creating a “certainty score” from the votes in random forests?

I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set ...
33
votes
6answers
4k views

Data mining: How should I go about finding the functional form?

I'm curious about repeatable procedures that can be used to discover the functional form of the function y = f(A, B, C) + error_term where my only input is a set of ...
26
votes
3answers
10k views

Topic models and word co-occurrence methods

Popular topic models like LDA usually cluster words that tend to co-occur together into the same topic (cluster). What is the main difference between such topic models, and other simple co-...
21
votes
1answer
2k views

Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
19
votes
6answers
10k views

Programmer looking to break into machine learning field

I am a software developer (mostly .NET and Python about 5 years experience). What can I do to help me get a job in the machine learning field or really anything that will get me started in that field? ...
14
votes
2answers
12k views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
26
votes
3answers
24k views

Unsupervised, supervised and semi-supervised learning

In the context of machine learning, what is the difference between unsupervised learning supervised learning and semi-supervised learning? And what are some of the main algorithmic approaches to ...
24
votes
1answer
37k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
14
votes
3answers
9k views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
24
votes
5answers
25k views

Machine learning algorithms to handle missing data

I am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the ...
17
votes
6answers
8k views

Is hyperparameter tuning on sample of dataset a bad idea?

I have a dataset of 140000 examples and 30 features for which I am training several classifiers for a binary classification (SVM, Logistic Regression, Random Forest etc) In many cases hyperparameter ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
19
votes
1answer
2k views

Justifying and choosing a proper scoring rule

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
10
votes
1answer
7k views

Scikit Binomial Deviance Loss Function

This is scikit GradientBoosting's binomial deviance loss function, ...
7
votes
1answer
2k views

When is a proper scoring rule a better estimate of generalization in a classification setting?

A typical approach to solving a classification problem is to identify a class of candidate models, and then perform model selection using some procedure like cross validation. Typically one selects ...
16
votes
2answers
14k views

Meaning of a neural network as a black-box?

I often hear people talking about neural networks as something as a black-box that you don't understand what it does or what they mean. I actually I can't understand what they mean by that! If you ...
12
votes
3answers
2k views

Is PCA optimization convex?

The objective function of Principal Component Analysis (PCA) is minimizing the reconstruction error in L2 norm (see section 2.12 here. Another view is trying to maximize the variance on projection. We ...
13
votes
1answer
731 views

Why can't a single ReLU learn a ReLU?

As a follow-up to My neural network can't even learn Euclidean distance I simplified even more and tried to train a single ReLU (with random weight) to a single ReLU. This is the simplest network ...
12
votes
5answers
5k views

Recursive (online) regularised least squares algorithm

Can anyone point me in the direction of an online (recursive) algorithm for Tikhonov Regularisation (regularised least squares)? In an offline setting, I would calculate $\hat\beta=(X^TX+λI)^{−1}X^TY$...
25
votes
3answers
32k views

How do you Interpret RMSLE (Root Mean Squared Logarithmic Error)?

I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem ...
29
votes
3answers
2k views

Utility of feature-engineering : Why create new features based on existing features?

I often see people create new features based on existing features on a machine learning problem. For example, here : https://triangleinequality.wordpress.com/2013/09/08/basic-feature-engineering-with-...
14
votes
4answers
6k views

What does “degree of freedom” mean in neural networks?

In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that ...
13
votes
1answer
873 views

Is the machine learning community abusing “conditioned on” and “parametrized by”?

Say, $X$ is dependent on $\alpha$. Rigorously speaking, if $X$ and $\alpha$ are both random variables, we could write $p(X\mid\alpha)$; however, if $X$ is a random variable and $\alpha$ is a ...
12
votes
1answer
5k views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
5
votes
2answers
702 views

Is dimensionality reduction almost always useful for classification?

Is singular value decomposition almost always useful in practice for enhancing the predicative power of a trained classification model? E.x. A dataset for classification has 20,000 features. Run SVD ...
9
votes
1answer
4k views

What is “baseline” in precision recall curve

I'm trying to understand precision recall curve, I understand what precision and recall are but the thing I don't understand is the "baseline" value. I was reading this link https://classeval....
7
votes
1answer
1k views

Cross-validation for timeseries data with regression

I am familiar with "regular" cross-validation, but now I want to make timeseries predictions while using cross-validation with a simple linear regression function. I write down a simple example, to ...
6
votes
1answer
645 views

the relationship between maximizing the likelihood and minimizing the cross-entropy

There is a statement that maximizing the likelihood is equivalent to minimizing the cross-entropy. Are there any proof for this statement?
5
votes
1answer
7k views

Is it possible for test error to be lower than training error

Is it possible to have test error lower than training error? I have a classification problem with 2000 samples, 500 of which are positives, 1500 are negatives. I split my data into 70% training data, ...
18
votes
3answers
3k views

Is a decision stump a linear model?

Decision stump is a decision tree with only one split. It can also be written as a piecewise function. For example, assume $x$ is a vector, and $x_1$ is the first component of $x$, in regression ...
3
votes
2answers
2k views

RMSE - where this evaluation metric came from?

Does anyone know where this metric came from ? Can someone bring article references or something like this? Im actually wondering if there's any mathematical concept or any way to demonstrate ...
2
votes
2answers
1k views

Machine learning feature encoding

I'm new to Machine Learning. I've just finished the Coursera course. :) And for my first practical attempt I wanted to "analyse" a local used cars selling website in order to compose a modal that ...
1
vote
1answer
544 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files(rough frequency of usage) in a filesystem (a company server on which 100s of company employees are active) in near future (say the next 1 ...
1
vote
1answer
298 views

Difference between linear regression and neural network

I am obviously confused with terms, and different concepts behind it. Each websites gives different intuitions. With all intuitions my brain is full of confusion now. Please help me to address what is ...
12
votes
4answers
2k views

Fast alternatives to the EM algorithm

Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
6
votes
4answers
523 views

Which data are used at each step of Stochastic Gradient Boosting? Subsample of the original training set or gradient of the loss function?

The bag.fraction parameter in SGB controls the size of the random subsample of the original training set on which each successive weak learner is fitted: At each ...
5
votes
2answers
6k views

Auto-regression versus linear regression of x(t)-with-t for modelling time series

What difference precisely does autoregression (for AR(p), p=1,2,...) have when compared to linear regression of that time series random variable w.r.t time axis? Explanation with diagrams clarifying ...