Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
11
votes
2answers
13k views

Why does PCA maximize total variance of the projection?

Christopher Bishop writes in his book Pattern Recognition and Machine Learning a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
17
votes
4answers
8k views

Is KNN a discriminative learning algorithm?

It seems that KNN is a discriminative learning algorithm but I can't seem to find any online sources confirming this. Is KNN a discriminative learning algorithm?
11
votes
2answers
2k views

Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
29
votes
3answers
10k views

In boosting, why are the learners “weak”?

See also a similar question on stats.SE. In boosting algorithms such as AdaBoost and LPBoost it is known that the "weak" learners to be combined only have to perform better than chance to be useful, ...
18
votes
2answers
8k views

What is “baseline” in precision recall curve

I'm trying to understand precision recall curve, I understand what precision and recall are but the thing I don't understand is the "baseline" value. I was reading this link https://classeval....
25
votes
2answers
2k views

Should final (production ready) model be trained on complete data or just on training set?

Suppose I trained several models on training set, choose best one using cross validation set and measured performance on test set. So now I have one final best model. Should I retrain it on my all ...
10
votes
1answer
560 views

What justifies this calculation of the derivative of a matrix function?

In Andrew Ng's machine learning course, he uses this formula: $\nabla_A tr(ABA^TC) = CAB + C^TAB^T$ and he does a quick proof which is shown below: $\nabla_A tr(ABA^TC) \\ = \nabla_A tr(f(A)A^TC) \\...
12
votes
1answer
5k views

How does gradient boosting calculate probability estimates?

I have been trying to understand gradient boosting reading various blogs, websites and trying to find my answer by looking through for example the XGBoost source code. However, I cannot seem to find ...
12
votes
1answer
2k views

What is the meaning of the axes in t-SNE?

I'm currently trying to wrap my head around the t-SNE math. Unfortunately, there is still one question I can't answer satisfactorily: What is the actual meaning of the axes in a t-SNE graph? If I were ...
10
votes
0answers
1k views

Machine learning self-learning book? [duplicate]

Possible Duplicate: Machine learning cookbook / reference card / cheatsheet? I wonder if there is a good self-learning textbook for machine learning? I am particularly looking for those in ...
6
votes
1answer
95 views

Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

Among others on here, Frank Harrell is adamant about using proper scoring rules to assess classifiers. This makes sense. If we have 500 $0$s with $P(1)\in[0.45, 0.49]$ and 500 $1$s with $P(1)\in[0.51, ...
5
votes
2answers
1k views

Graphical interpretation of LASSO

I've a question regarding to the graphical intuition of the LASSO. I'm understanding why the lasso produce zero coefficient in case of intersecting a corner of the diamond. But I don't understand the ...
3
votes
1answer
235 views

Neural Networks input data normalization and centering

I'm learning Neural Networks and I grasped the algebra behind them. I'm now interested in understanding how normalization and centering of the input data affect them. In my personal learning project (...
148
votes
2answers
157k views

A list of cost functions used in neural networks, alongside applications

What are common cost functions used in evaluating the performance of neural networks? Details (feel free to skip the rest of this question, my intent here is simply to provide clarification on ...
104
votes
8answers
81k views

Objective function, cost function, loss function: are they the same thing?

In machine learning, people talk about objective function, cost function, loss function. Are they just different names of the same thing? When to use them? If they are not always refer to the same ...
88
votes
2answers
103k views

tanh activation function vs sigmoid activation function

The tanh activation function is: $$tanh \left( x \right) = 2 \cdot \sigma \left( 2 x \right) - 1$$ Where $\sigma(x)$, the sigmoid function, is defined as: $$\sigma(x) = \frac{e^x}{1 + e^x}$$. ...
69
votes
4answers
39k views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
78
votes
7answers
29k views

Euclidean distance is usually not good for sparse data (and more general case)?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
63
votes
4answers
29k views

What're the differences between PCA and autoencoder?

Both PCA and autoencoder can do demension reduction, so what are the difference between them? In what situation I should use one over another?
56
votes
1answer
75k views

Performance metrics to evaluate unsupervised learning

With respect to the unsupervised learning (like clustering), are there any metrics to evaluate performance?
31
votes
5answers
10k views

How to deal with hierarchical / nested data in machine learning

I'll explain my problem with an example. Suppose you want to predict the income of an individual given some attributes: {Age, Gender, Country, Region, City}. You have a training dataset like so <...
17
votes
12answers
32k views

Best books for an introduction to statistical data analysis?

I bought this book: How to Measure Anything: Finding the Value of Intangibles in Business and Head First Data Analysis: A Learner's Guide to Big Numbers, Statistics, and Good Decisions What ...
30
votes
2answers
27k views

How to statistically compare the performance of machine learning classifiers?

Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing ...
21
votes
2answers
7k views

Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
31
votes
3answers
7k views

Cross-validation including training, validation, and testing. Why do we need three subsets?

I have a question regarding the Cross-validation process. I am in the middle of a course of the Machine Learning on the Cursera. One of the topic is about the Cross-validation. I found it slightly ...
23
votes
8answers
20k views

Perform K-means (or its close kin) clustering with only a distance matrix, not points-by-features data

I want to perform K-means clustering on objects I have, but the objects aren't described as points in space, i.e. by objects x features dataset. However, I am able ...
28
votes
2answers
36k views

10-fold Cross-validation vs leave-one-out cross-validation

I'm doing nested cross-validation. I have read that leave-one-out cross-validation can be biased (don't remember why). Is it better to use 10-fold cross-validation or leave-one-out cross-validation ...
16
votes
1answer
7k views

Including Interaction Terms in Random Forest

Suppose we have a response Y and predictors X1,....,Xn. If we were to try to fit Y via a linear model of X1,....,Xn, and it just so happened that the true relationship between Y and X1,...,Xn wasn't ...
14
votes
2answers
7k views

Use of nested cross-validation

Scikit Learn's page on Model Selection mentions the use of nested cross-validation: ...
23
votes
1answer
3k views

Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
13
votes
3answers
3k views

Is PCA optimization convex?

The objective function of Principal Component Analysis (PCA) is minimizing the reconstruction error in L2 norm (see section 2.12 here. Another view is trying to maximize the variance on projection. We ...
14
votes
3answers
7k views

Perform linear regression, but force solution to go through some particular data points

I know how to perform a linear regression on a set of points. That is, I know how to fit a polynomial of my choice, to a given data set, (in the LSE sense). However, what I do not know, is how to ...
20
votes
3answers
4k views

Why AUC =1 even classifier has misclassified half of the samples?

I am using a classifier which returns probabilities. To calculate AUC, I am using pROC R-package. The output probabilities from classifier are: ...
12
votes
1answer
5k views

How to fit weights into Q-values with linear function approximation

In reinforcement learning, linear function approximation is often used when large state spaces are present. (When look up tables become unfeasible.) The form of the $Q-$value with linear function ...
12
votes
2answers
8k views

What is the difference between 'regular' linear regression and deep learning linear regression?

I want to know the difference between linear regression in a regular machine learning analysis and linear regression in "deep learning" setting. What algorithms are used for linear regression in deep ...
9
votes
3answers
2k views

Expected best performance possible on a data set

Say I have a simple machine learning problem like a classification. With some benchmarks in vision or audio recognition, I, as a human, are a very good classifier. I therefore have an intuition on how ...
9
votes
1answer
1k views

the relationship between maximizing the likelihood and minimizing the cross-entropy

There is a statement that maximizing the likelihood is equivalent to minimizing the cross-entropy. Are there any proof for this statement?
15
votes
4answers
637 views

What *is* an Artificial Neural Network?

As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
5
votes
2answers
2k views

Detecting Bimodal Distribution

I have histograms of audio signals where they have bimodal "normal" distribution. What I want to do is to detect these subpopulations inorder to have a threshold, this is meant to divide the values ...
3
votes
2answers
2k views

How to bootstrap panel data?

I'm fitting some machine learning algorithms (e.g. SVM) on my panel data. It's taking too long for my entire dataset, so I'm considering generating smaller samples from bootstrapping then fit the SVM ...
5
votes
1answer
5k views

Time steps in Keras LSTM

My understanding of time-series LSTM training is that the recurrent cell gets unrolled to a specified length (num_steps), and parameter updates are back-propagated ...
2
votes
1answer
5k views

Decision boundaries and Gaussian density functions

This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question: Assume a two-class problem with equal a priori class probabilities and Gaussian class-...
4
votes
2answers
4k views

Why is my high degree polynomial regression model suddenly unfit for the data?

I'm building a ridge regression model in scikit-learn and trying to find the optimal degree polynomial to use. The data I'm working with is a fairly predictable time series of hourly traffic volumes, ...
71
votes
9answers
80k views

What algorithm should I use to detect anomalies on time-series?

Background I'm working in Network Operations Center, we monitor computer systems and their performance. One of the key metrics to monitor is a number of visitors\customers currently connected to our ...
98
votes
11answers
33k views

Explain “Curse of dimensionality” to a child

I heard many times about curse of dimensionality, but somehow I'm still unable to grasp the idea, it's all foggy. Can anyone explain this in the most intuitive way, as you would explain it to a child,...
74
votes
11answers
44k views

Having a job in data-mining without a PhD

I've been very interested in data-mining and machine-learning for a while, partly because I majored in that area at school, but also because I am truly much more excited trying to solve problems that ...
42
votes
4answers
33k views

How does rectilinear activation function solve the vanishing gradient problem in neural networks?

I found rectified linear unit (ReLU) praised at several places as a solution to the vanishing gradient problem for neural networks. That is, one uses max(0,x) as activation function. When the ...
33
votes
6answers
20k views

How to get started with neural networks

I'm completely new to neural networks but highly interested in understanding them. However it's not easy at all to get started. Could anyone recommend a good book or any other kind of resource? Is ...
30
votes
8answers
41k views

What math subjects would you suggest to prepare for data mining and machine learning?

I'm trying to put together a self-directed math curriculum to prepare for learning data mining and machine learning. This is motivated by starting Andrew Ng's machine learning class on Coursera and ...
38
votes
4answers
26k views

Is a strong background in maths a total requisite for ML?

I'm starting to want to advance my own skillset and I've always been fascinated by machine learning. However, six years ago instead of pursuing this I decided to take a completely unrelated degree to ...

1 2
3
4 5
30