Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
7
votes
3answers
9k views

What balancing method can I apply to a imbalanced data set?

I'm trying to solve one classification problem from the UCI database repository. Unfortunately (or fortunately), I've noticed that my dataset is imbalanced. I've structured the data as 5 classes, ...
7
votes
3answers
2k views

Is a lower training accuracy possible in overfitting (one class SVM)

I am using the heart_scale data from LibSVM. The original data includes 13 features, but I only used 2 of them in order to plot the distributions in a figure. Instead of training the binary classifier,...
6
votes
1answer
665 views

What do “real values” refer to in supervised classification?

I'm using supervised classification algorithms from mlpy to classify things into two groups for a question-answering system. I don't really know how these algorithms work, but they seem to be doing ...
5
votes
2answers
443 views

Graphical interpretation of LASSO

I've a question regarding to the graphical intuition of the LASSO. I'm understanding why the lasso produce zero coefficient in case of intersecting a corner of the diamond. But I don't understand the ...
4
votes
2answers
2k views

Detecting Bimodal Distribution

I have histograms of audio signals where they have bimodal "normal" distribution. What I want to do is to detect these subpopulations inorder to have a threshold, this is meant to divide the values ...
3
votes
1answer
807 views

How to do a bias-variance analysis on a machine learning modelling process

I searched on topics of the bias and variance trade-off and got back lots of questions with different levels of response. The information is scattering too much and unsystematic to answer my own ...
2
votes
1answer
440 views

Graphical path Coordinate Descent in case of semi-differentiable functions such as Lasso

I am trying to understand how the graphical solution path to the optimum would look in the case of Lasso Regression. I can find only Pictures for the differentiable or non differentiable case. The ...
1
vote
1answer
3k views

Time steps in Keras LSTM

My understanding of time-series LSTM training is that the recurrent cell gets unrolled to a specified length (num_steps), and parameter updates are back-propagated ...
107
votes
2answers
112k views

Gradient Boosting Tree vs Random Forest

Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there ...
86
votes
4answers
56k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
67
votes
9answers
74k views

What algorithm should I use to detect anomalies on time-series?

Background I'm working in Network Operations Center, we monitor computer systems and their performance. One of the key metrics to monitor is a number of visitors\customers currently connected to our ...
70
votes
7answers
25k views

Euclidean distance is usually not good for sparse data?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
48
votes
5answers
38k views

Is it necessary to scale the target value in addition to scaling features for regression analysis?

I'm building regression models. As a preprocessing step, I scale my feature values to have mean 0 and standard deviation 1. Is it necessary to normalize the target values also?
52
votes
3answers
29k views

Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...
47
votes
4answers
39k views

Why not approach classification through regression?

Some material I've seen on machine learning said that it's a bad idea to approach a classification problem through regression. But I think it's always possible to do a continuous regression to fit the ...
37
votes
3answers
29k views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
29
votes
7answers
13k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
25
votes
3answers
34k views

How to judge if a supervised machine learning model is overfitting or not?

Can anyone tell me how to judge if a supervised machine learning model is overfitting or not? If I don't have an external validation dataset, I want to know if I can use ROC of 10 fold cross ...
36
votes
3answers
21k views

Creating a “certainty score” from the votes in random forests?

I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set ...
32
votes
4answers
33k views

What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
36
votes
3answers
42k views

Things to consider about masters programs in statistics

It is admission season for graduate schools. I (and many students like me) am now trying to decide which statistics program to pick. What are some things those of you who work with statistics suggest ...
26
votes
3answers
10k views

Topic models and word co-occurrence methods

Popular topic models like LDA usually cluster words that tend to co-occur together into the same topic (cluster). What is the main difference between such topic models, and other simple co-...
21
votes
1answer
2k views

Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
19
votes
6answers
10k views

Programmer looking to break into machine learning field

I am a software developer (mostly .NET and Python about 5 years experience). What can I do to help me get a job in the machine learning field or really anything that will get me started in that field? ...
16
votes
3answers
12k views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
27
votes
3answers
24k views

Unsupervised, supervised and semi-supervised learning

In the context of machine learning, what is the difference between unsupervised learning supervised learning and semi-supervised learning? And what are some of the main algorithmic approaches to ...
24
votes
1answer
39k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
14
votes
3answers
9k views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
24
votes
5answers
26k views

Machine learning algorithms to handle missing data

I am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the ...
18
votes
6answers
8k views

Is hyperparameter tuning on sample of dataset a bad idea?

I have a dataset of 140000 examples and 30 features for which I am training several classifiers for a binary classification (SVM, Logistic Regression, Random Forest etc) In many cases hyperparameter ...
18
votes
4answers
22k views

How to recode categorical variable into numerical variable when using SVM or Neural Network

To use SVM or Neural Network it needs to transform (encode) categorical variables into numeric variables, the normal method in this case is to use 0-1 binary values with the k-th categorical value ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
10
votes
1answer
8k views

Scikit Binomial Deviance Loss Function

This is scikit GradientBoosting's binomial deviance loss function, ...
7
votes
1answer
2k views

When is a proper scoring rule a better estimate of generalization in a classification setting?

A typical approach to solving a classification problem is to identify a class of candidate models, and then perform model selection using some procedure like cross validation. Typically one selects ...
18
votes
2answers
15k views

Meaning of a neural network as a black-box?

I often hear people talking about neural networks as something as a black-box that you don't understand what it does or what they mean. I actually I can't understand what they mean by that! If you ...
12
votes
3answers
2k views

Is PCA optimization convex?

The objective function of Principal Component Analysis (PCA) is minimizing the reconstruction error in L2 norm (see section 2.12 here. Another view is trying to maximize the variance on projection. We ...
16
votes
4answers
6k views

What does “degree of freedom” mean in neural networks?

In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that ...
29
votes
3answers
2k views

Utility of feature-engineering : Why create new features based on existing features?

I often see people create new features based on existing features on a machine learning problem. For example, here : https://triangleinequality.wordpress.com/2013/09/08/basic-feature-engineering-with-...
25
votes
4answers
33k views

How do you Interpret RMSLE (Root Mean Squared Logarithmic Error)?

I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem ...
12
votes
1answer
5k views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
6
votes
2answers
766 views

Is dimensionality reduction almost always useful for classification?

Is singular value decomposition almost always useful in practice for enhancing the predicative power of a trained classification model? E.x. A dataset for classification has 20,000 features. Run SVD ...
7
votes
1answer
2k views

Cross-validation for timeseries data with regression

I am familiar with "regular" cross-validation, but now I want to make timeseries predictions while using cross-validation with a simple linear regression function. I write down a simple example, to ...
7
votes
1answer
732 views

the relationship between maximizing the likelihood and minimizing the cross-entropy

There is a statement that maximizing the likelihood is equivalent to minimizing the cross-entropy. Are there any proof for this statement?
5
votes
1answer
7k views

Is it possible for test error to be lower than training error

Is it possible to have test error lower than training error? I have a classification problem with 2000 samples, 500 of which are positives, 1500 are negatives. I split my data into 70% training data, ...
19
votes
3answers
3k views

Is a decision stump a linear model?

Decision stump is a decision tree with only one split. It can also be written as a piecewise function. For example, assume $x$ is a vector, and $x_1$ is the first component of $x$, in regression ...
14
votes
3answers
504 views

What *is* an Artificial Neural Network?

As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
3
votes
2answers
2k views

RMSE - where this evaluation metric came from?

Does anyone know where this metric came from ? Can someone bring article references or something like this? Im actually wondering if there's any mathematical concept or any way to demonstrate ...
2
votes
2answers
1k views

Machine learning feature encoding

I'm new to Machine Learning. I've just finished the Coursera course. :) And for my first practical attempt I wanted to "analyse" a local used cars selling website in order to compose a modal that ...
1
vote
1answer
414 views

Difference between linear regression and neural network

I am obviously confused with terms, and different concepts behind it. Each websites gives different intuitions. With all intuitions my brain is full of confusion now. Please help me to address what is ...
1
vote
1answer
608 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files(rough frequency of usage) in a filesystem (a company server on which 100s of company employees are active) in near future (say the next 1 ...