Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

5,439 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
1k views

How do I find multiple change points in an online dataset?

I am trying to develop a Python based script connected to a SQLite3 database to identify distinct system changepoints in an "online" datastream. Changepoint must be identified in less than 2 minutes ...
4
votes
0answers
2k views

How to know when to use Kernel SVM and not Linear SVM?

If I have more than 3 features in a dataset, then I can't visualize them to see if my classes are scattered in a non linear fashion. So how do I know when is the right way to use linear model with non-...
4
votes
0answers
789 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
1answer
789 views

Difference between training and test data distribution

The basic assumption in machine learning is training and test data follows same distribution. But in reality this is highly unlikely. Covariate shift address this issue in which training and test ...
4
votes
1answer
444 views

A statistical test to measure the importance of features?

I'm currently trying to assess importance of the features for my classifier. The situation is the following: first I train my classifier with all of the features I have and tested on a test set . Then ...
4
votes
0answers
233 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
4
votes
0answers
1k views

How to draw plot of the values of decision function of multi class svm versus another arbitrary values?

I am trying to draw a plot of the decision function ($f(x)=sign(wx+b)$ which can be obtain by fit$decision.values in R using the svm function of e1071 package) versus another arbitrary values. From ...
4
votes
0answers
487 views

Decomposing the non-deterministic transition functions in non-Markov decision processes into several deterministic transition functions

Problems in reinforcement learning are commonly modeled as Markov decision processes (MDPs). One essential part of MDPs is the transition function $T: S \times A \times S \rightarrow [0, 1] \in \...
4
votes
0answers
247 views

Modelling of probabilistic vs deterministic systems

The learning problem in Statistical Learning Theory is defined as: $$ R(f) = \int_{X,Y} L(y, f(x))P(x,y)\mathrm{d}x\mathrm{d}y $$ where $R(f)$ is the expected risk $L$ is the loss function $P(x, y)...
4
votes
0answers
876 views

When to use the Kappa statistic evaluation metric?

Can someone tell me when is it appropriate to use the Kappa statistic? Also why to use it when one can use Area Under the ROC curve? Or even the Area under the precision-recall curve? So what are the ...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
596 views

How to understand kernel functions and how to choose a suitable kernel?

I am trying to describe my understand of kernels in the Support Vector Machine(SVM) and why some of them are more popular, but I am not sure if I misunderstand these concepts: 1) There are a large ...
4
votes
0answers
1k views

What are appropriate validation methods for a Bayesian network model with low sample size?

I am currently using a Bayesian network model with 20 variables and 210 data points, with 15 locations measured at 14 different time points each. There are also some restrictions on what types of ...
4
votes
0answers
249 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
121 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
899 views

Learning to Rank: query-dependent vs. query-independent features

I've been doing some reading about learning to rank - specifically lambdaMART - and one thing I am confused about is the role of features. When training a model, should one only use query-dependent ...
4
votes
0answers
172 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
4
votes
0answers
263 views

Time series prediction where each datapoint has a sequence

I am Computer Science major, and new to stats, so please bear with me and point me to the right direction if what I'm asking is pretty obvious. I have a dataset, where each data point consists of <...
4
votes
0answers
470 views

Reducing size of dataset to a fixed size - retaining maximum information in all dimensions

I was wondering about about the following problem: I have a set of $N=10^5$ observations with dimensionality $D=2$, and I would like to reduce it to a set of size with $M=10^3$, or some other $(M \ll ...
4
votes
0answers
362 views

Linear Discriminant Analysis: Using subject as classification

I have a problem where I need to identify from which subject a particular set of data points came. More specifically, my problem is that I need to demonstrate that my subjects (N=9) can be ...
4
votes
0answers
4k views

Minimum training sample size required for a classifier

What is the best method to determine the minimum number of training samples required for a classifier? I am only comparing one classifier (four class problem), discriminant function analysis (DFA) ...
4
votes
0answers
1k views

Prediction using SVD and Fisher's linear discriminant

Where can I get an explanation of the procedure used when making a prediction using SVD? Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
4
votes
0answers
3k views

How to train SVM correctly on a 1D dataset

I am trying to use svmtrain (Statistic Toolbox) to train a linear (2 class) SVM on a 1D feature vectors. The features are not fully separable and the classes intersect. The naive approach would be ...
4
votes
0answers
92 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
2k views

SVM classifier (with soft-margin) implementation in R, gamma value and quadprog

I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form : $$min_b \frac{1}{2} b^...
4
votes
0answers
584 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
667 views

Anomaly detection in user behaviour using hidden Markov models

I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the ...
4
votes
0answers
218 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
4
votes
0answers
841 views

Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...
4
votes
0answers
3k views

Adjusting the classification threshold of Naive Bayes

I've been involved in a machine learning project recently and am now in the process of writing the project up for a paper submission. We used the naive bayes classifier on the project and developed a ...
4
votes
1answer
808 views

Formula for marginal probability in CRF++

On the website for CRF++ http://crfpp.sourceforge.net/ they mention that marginal probabilities can be output for each possible label. My question is, in CRF theory, what's the formula for this ...
4
votes
1answer
416 views

Observation symbols for training a set of HMMs

If we are to classify 2 separate classes/actions using HMMs, we design 2 separate HMMs (one for each class). Do they share same set or a different set of observations-symbols for each of the HMM? If ...
4
votes
0answers
312 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
1answer
761 views

Why we don't normalize the images?

I was watching the video from this stanford course on convolutional neural nets where the professor says (at 28:59) 'we do zero-mean the pixel values in image but we do not normalize the pixel values ...
4
votes
1answer
152 views

Sampling a test set from global spatial data

The basis of testing the accuracy of any machine learning algorithm is to test the trained algorithm on data that it has never seen before. The usual approach to sample the test set is to just ...
4
votes
2answers
4k views

multiclass classification having class imbalance with Gradient Boosting Classifier

I am using Abalon data for classification from UCI(https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data). I have scaled data and used TSNE for visualization. ...
4
votes
1answer
473 views

Semi-supervised method for identifying states and state durations in a time series for anomaly detection

I am developing a semi-supervised method for identifying anomalies in a time series with multiple states. Let's consider this example time series in which there are two states e.g. state 1 and 2 with ...
4
votes
1answer
800 views

Test for Statistical Significance in the Accuracy of a Machine Learning System

I have what I imagine is an elementary question about evaluating statistical significance, but while I know a lot about probability I can't t-test my way out of a paper bag. From here I'm hoping to ...
3
votes
1answer
39 views

Does using a random train-test split lead to data leakage?

I am trying to understand data leakage in modeling practice. If we had a dataset of patient instances from 2000-2018 (with all patient visits included), and used a randomly selected train-test split (...
3
votes
0answers
46 views
+50

ANOVA vs. mixed models

I'm confused between the differences between x-way/mixed ANOVA models and mixed models. Is there a difference? If so, what is the difference and why?
3
votes
0answers
25 views

Modeling Expensive Functions Using ML (Q1), Is $r^2$ Appropriate?

I may end up asking several follow-up questions in different posts, hence the broad view of my problem. Context and Background I have a function which is very expensive to calculate. Imagine ...
3
votes
0answers
42 views

Statistical data vs Precise data

I have a dataset in which I assign descriptive-statistical data of the geographical zone to each person of the dataset (obviously, the person belongs to that specific zone). For example, in a given ...
3
votes
0answers
39 views

How can I prove the ill-effects of binning/discretization?

There is a binary classification model built where there is grouping of continuous variables into arbitrary ranges which I am told is to include a good number of outliers in the data set. How do I ...
3
votes
0answers
31 views

Bootstrap Resampling vs Repeated K-Fold Cross-Validation for Confidence Intervals

Why is bootstrap resampling with replacement used to construct confidence intervals over repeated K-fold cross-validation? Isn't it valid to use 10-fold CV repeated 10 times, where we garner 100 data ...
3
votes
0answers
46 views

Why can't we accurately compute covariance matrix in high dimensions?

I am reading pg 651 of Elements of Statistical Learning,where is says: "The simplest form of regularization assumes that the features are independent within each class, that is, the within-class ...
3
votes
1answer
81 views

What exactly does a proper scoring rule want to do?

I will adapt an excellent simulation by our Stéphane Laurent for this question. ...
3
votes
0answers
49 views

Ways to estimate error in a machine learning algorithm's predictions by outputting a probability distribution

I want to make it so that my machine-learning algorithm, when given an input, outputs the parameters of a Gaussian distribution, with the goal of getting an expected error on the prediction. The ...
3
votes
0answers
129 views

Distribution of OLS predictions

Suppose: $y = X\beta + \varepsilon$, with $\varepsilon \sim (0, \Omega) \Rightarrow y|X \sim (X \beta, \Omega)$ $\hat{\beta}_{ols} = (X'X)^{-1}X'Y = \beta +(X'X)^{-1}X'\varepsilon \sim (\beta, \Sigma)...
3
votes
0answers
20 views

Benefits of saturation in activation functions

It's known that saturation of activation functions in neural networks leads to vanishing gradients or dead units, so modern practice often avoids them, instead opting for e.g. ReLUs, Leaky ReLUs and ...
3
votes
0answers
38 views

Why is vectorization called so?

As I understand a vector is anything with a dimension of n x 1. While a matrix is anything with a dimension of n x n. Where n can be any number. So, to my knowledge, when we convert a scalar ...