Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
16
votes
3answers
10k views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
25
votes
5answers
28k views

Machine learning algorithms to handle missing data

I am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
20
votes
6answers
11k views

Is hyperparameter tuning on sample of dataset a bad idea?

I have a dataset of 140000 examples and 30 features for which I am training several classifiers for a binary classification (SVM, Logistic Regression, Random Forest etc) In many cases hyperparameter ...
11
votes
1answer
9k views

Scikit Binomial Deviance Loss Function

This is scikit GradientBoosting's binomial deviance loss function, ...
9
votes
1answer
2k views

When is a proper scoring rule a better estimate of generalization in a classification setting?

A typical approach to solving a classification problem is to identify a class of candidate models, and then perform model selection using some procedure like cross validation. Typically one selects ...
11
votes
2answers
5k views

machine learning techniques for longitudinal data

I was wondering if there were any machine learning techniques (unsupervised) for modelling longitudinal data? I've always used mixed effects models (mostly non-linear) but I was wondering if there are ...
33
votes
4answers
41k views

How do you Interpret RMSLE (Root Mean Squared Logarithmic Error)?

I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem ...
18
votes
4answers
8k views

What does “degree of freedom” mean in neural networks?

In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that ...
30
votes
3answers
3k views

Utility of feature-engineering : Why create new features based on existing features?

I often see people create new features based on existing features on a machine learning problem. For example, here : https://triangleinequality.wordpress.com/2013/09/08/basic-feature-engineering-with-...
13
votes
1answer
11k views

Can support vector machine be used in large data?

With the limited knowledge I have on SVM, it is good for a short and fat data matrix $X$, (lots of features, and not too many instances), but not for big data. I understand one reason is the Kernel ...
13
votes
1answer
7k views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
8
votes
1answer
3k views

Cross-validation for timeseries data with regression

I am familiar with "regular" cross-validation, but now I want to make timeseries predictions while using cross-validation with a simple linear regression function. I write down a simple example, to ...
18
votes
3answers
8k views

ROC vs Precision-recall curves on imbalanced dataset

I just finished reading this discussion. They argue that PR AUC is better than ROC AUC on imbalanced dataset. For example, we have 10 samples in test dataset. 9 samples are positive and 1 is ...
9
votes
1answer
9k views

What are the implications of scaling the features to xgboost?

Doing research about the xgboost algorithm I went through the documentation. I have heard that xgboost does not care much about the scale of the input features In this approach trees are ...
6
votes
2answers
1k views

Is dimensionality reduction almost always useful for classification?

Is singular value decomposition almost always useful in practice for enhancing the predicative power of a trained classification model? E.x. A dataset for classification has 20,000 features. Run SVD ...
6
votes
1answer
9k views

Is it possible for test error to be lower than training error

Is it possible to have test error lower than training error? I have a classification problem with 2000 samples, 500 of which are positives, 1500 are negatives. I split my data into 70% training data, ...
3
votes
1answer
542 views

Does changing the parameter search space after nested CV introduce optimistic bias?

Suppose I am fitting a Ridge and I decide to search a parameter space for c:[1,2,3]. I perform nested CV on my whole dataset and find the performance not so great. I therefore expand my search space ...
7
votes
3answers
2k views

Is a lower training accuracy possible in overfitting (one class SVM)

I am using the heart_scale data from LibSVM. The original data includes 13 features, but I only used 2 of them in order to plot the distributions in a figure. Instead of training the binary classifier,...
19
votes
3answers
3k views

Is a decision stump a linear model?

Decision stump is a decision tree with only one split. It can also be written as a piecewise function. For example, assume $x$ is a vector, and $x_1$ is the first component of $x$, in regression ...
13
votes
4answers
2k views

Fast alternatives to the EM algorithm

Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
6
votes
2answers
8k views

Auto-regression versus linear regression of x(t)-with-t for modelling time series

What difference precisely does autoregression (for AR(p), p=1,2,...) have when compared to linear regression of that time series random variable w.r.t time axis? Explanation with diagrams clarifying ...
3
votes
2answers
550 views

Do we always assume cross entropy cost function for logistic regression solution unless stated otherwise?

I am using Matlab glmfit for logistic regression. Now I know that usually people use the cross entropy to evaluate the error in predictions against the true labels ( which different than the linear ...
3
votes
2answers
2k views

RMSE - where this evaluation metric came from?

Does anyone know where this metric came from ? Can someone bring article references or something like this? Im actually wondering if there's any mathematical concept or any way to demonstrate ...
2
votes
1answer
952 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files(rough frequency of usage) in a filesystem (a company server on which 100s of company employees are active) in near future (say the next 1 ...
6
votes
4answers
576 views

Which data are used at each step of Stochastic Gradient Boosting? Subsample of the original training set or gradient of the loss function?

The bag.fraction parameter in SGB controls the size of the random subsample of the original training set on which each successive weak learner is fitted: At each ...
4
votes
1answer
4k views

Linear separability for a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ $...
2
votes
1answer
886 views

Gaussian is conjugate of Gaussian?

Someone told me that, Gaussian distribution is conjugate to distribution because a Gaussian times a Gaussian would still be Gaussian distribution ? Why is that ? Say the following situation: $X\sim N(...
9
votes
1answer
6k views

How to Identify Overfitting in Convolutional Neural network?

I understand that dropout is used to reduce over fitting in the network. This is a generalization technique. In convolutional neural network how can I identify overfitting? One situation that I can ...
3
votes
2answers
1k views

Trouble applying hidden Markov models

Edit: I updated the question to hopefully make it more easy to understand. I think it was overly complex. I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
2
votes
1answer
923 views

Deriving the Ridge Regression $\boldsymbol{\beta}\mid \mathbf{y}$ distribution

Apparently the estimate $\hat{\boldsymbol{\beta}}$ for ridge regression comes up as the mean or mode of the posterior distribution given by $f_{\boldsymbol{\beta}\mid \mathbf{y}}$. This is the ...
1
vote
1answer
165 views

Valid references on origins of Machine Learning, Statistical Learning and Data Mining

I know it's a rather debated question on Stack Exchange communities but let me explain the points of this question. I'm writing my capstone on Machine Learning and I need to clarify deeply, giving ...
0
votes
0answers
75 views

How to derive a ranking function by analysing feature correlations

I am analysing some employee details to find the efficiency of the employees. Ideally I want some rankings to rank them based on these features. My features include; current salary projects ...
193
votes
4answers
176k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
46
votes
6answers
4k views

What are the main theorems in Machine (Deep) Learning?

Al Rahimi has recently given a very provocative talk in NIPS 2017 comparing current Machine Learning to Alchemy. One of his claims is that we need to get back to theoretical developments, to have ...
54
votes
3answers
49k views

Clustering with K-Means and EM: how are they related?

I have studied algorithms for clustering data (unsupervised learning): EM, and k-means. I keep reading the following : k-means is a variant of EM, with the assumptions that clusters are ...
40
votes
4answers
76k views

Recall and precision in classification

I read some definitions of recall and precision, though it is every time in the context of information retrieval. I was wondering if someone could explain this a bit more in a classification context ...
48
votes
7answers
4k views

Where to start with statistics for an experienced developer

During the first half of 2015 I did the coursera course of Machine Learning (by Andrew Ng, GREAT course). And learned the basics of machine learning (linear regression, logistic regression, SVM, ...
48
votes
1answer
19k views

Understanding “almost all local minimum have very similar function value to the global optimum”

In a recent blog post by Rong Ge, it was said that: It is believed that for many problems including learning deep nets, almost all local minimum have very similar function value to the global ...
50
votes
1answer
72k views

How large should the batch size be for stochastic gradient descent?

I understand that stochastic gradient descent may be used to optimize a neural network using backpropagation by updating each iteration with a different sample of the training dataset. How large ...
51
votes
4answers
42k views

Recurrent vs Recursive Neural Networks: Which is better for NLP?

There are Recurrent Neural Networks and Recursive Neural Networks. Both are usually denoted by the same acronym: RNN. According to Wikipedia, Recurrent NN are in fact Recursive NN, but I don't really ...
45
votes
4answers
23k views

How are kernels applied to feature maps to produce other feature maps?

I am trying to understand the convolution part of convolutional neural networks. Looking at the following figure: I have no problems understanding the first convolution layer where we have 4 ...
32
votes
4answers
13k views

Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
28
votes
4answers
35k views

Number of features vs. number of observations

Are there any papers/books/ideas about the relationship between the number of features and the number of observations one needs to have to train a "robust" classifier? For example, assume I have 1000 ...
27
votes
4answers
33k views

Neural network with skip-layer connections

I am interested in regression with neural networks. Neural networks with zero hidden nodes + skip-layer connections are linear models. What about the same neural nets but with hidden nodes ? I am ...
22
votes
1answer
30k views

Discussion about overfit in xgboost

My set-up is the following: I am following the guidlines in "Applied Predictive Modelling". Thus I have filtered correlated features and end up with the following: 4900 data points in the training ...
23
votes
3answers
3k views

Sites for predictive modeling competitions

I participate in predictive modeling competitions on Kaggle, TunedIt, and CrowdAnalytix. I find that these sites are a good way to "work-out" for statistics/machine learning. Are there any other ...
24
votes
2answers
7k views

Understanding bias-variance tradeoff derivation

I am reading the chapter of bias-variance tradeoff of The elements of statistical learning and I have doubt in the formula at page 29. Let the data arise from a model such that $$ Y = f(x)+\epsilon$$ ...
24
votes
1answer
5k views

What are variational autoencoders and to what learning tasks are they used?

As per this and this answer, autoencoders seem to be a technique that uses neural networks for dimension reduction. I would like to additionally know what is a variational autoencoder (its main ...
24
votes
1answer
21k views

Should I make decisions based on micro-averaged or macro-averaged evaluation measures?

I ran a 10-fold cross validation on different binary classification algorithms, with the same dataset, and received both Micro- and Macro averaged results. It should be mentioned that this was a multi-...

1
3 4
5
6 7
30