Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
10
votes
3answers
1k views

How to compare the accuracy of two different models using statistical significance

I am working on time series prediction. I have two data sets $D1=\{x_1, x_2,....x_n\}$ and $D2=\{x_n+1, x_n+2, x_n+3,...., x_n+k\}$. I have three prediction models: $M1, M2, M3$. All of those model ...
0
votes
1answer
123 views

Is the zero a valid kernel?

It seems to me that if $c\geq0$ then $k(x,y)=c$ is a valid kernel since no rules are broken. However, one of the rules of constructing a kernel disagrees with this. The rule is $k(x,y)=ck'(x,y)$, ...
1
vote
1answer
27 views

Combining continuous and binary data in unsupervised learning

I am working on cluster detection in a data set consisting of housing data. Each data point has some continuous features, such as house size, and some discrete ones, such as the number of garages (0 ...
0
votes
1answer
26 views

Decision tree that fits a regression at leaf nodes

Is there any academic work on any Decision Tree that fits a regression at its final leaf node? For instance, suppose I have 100 features (X), and use them build a tree with 3 depths such that I have ...
0
votes
0answers
25 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertunned ML trees etc. ...
1
vote
1answer
128 views

On evaluating variational autoencoders with prior likelihood and reconstruction error

A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
3
votes
2answers
3k views

Understanding Add-1/Laplace smoothing with bigrams

I am working through an example of Add-1 smoothing in the context of NLP Say that there is the following corpus (start and end tokens included) ...
0
votes
0answers
20 views

MLE vs MAP hypothesis, performance

I have a question regarding the performance of Maximum a posteriori vs Maximum Likelihood Estimation hypothesis. If you only consider the training data. Can you then state that the MAP hypothesis ...
1
vote
0answers
13 views

What are the limitations of non-negative matrix factorisation when reducing the dimensions of a data set?

From what I understand NFM (non-negative matrix factorisation) is constrained by the factor that it only supports data sets with non-negative values when reducing the dimensions of a data set. ...
0
votes
0answers
13 views

How does ReLU provide non-linearity? [duplicate]

ReLU is linear or 0. How does this property help NN to learn higher degree polynomial function? If the output is always positive then it is same as linear and I can always write output as a linear ...
3
votes
1answer
871 views

What exactly is the mathematical definition of a classifier / classification algorithm?

I just started an intro machine learning course, and to get things better organized in my head, I was trying to come up with exactly what is needed to completely specify a classification algorithm. I ...
2
votes
0answers
20 views

Relationship between generalization error and prior variance

Suppose that we have a Bayesian linear regression with prior on the weights $p(w)=N(0,\sigma_{w}^{2})$ Our goal is to use that Bayesian linear regression for future predictions. In order to do that ...
-1
votes
1answer
77 views

Why am I getting accuracy of 100 percent using SVM

I am working on Credit card data set for fraud detection. When I apply SVM for it, I am getting the accuracy as 100 %. What might be going wrong here? Here is the code ...
0
votes
0answers
18 views

Order-insensitive variable-length sequence encoder?

While brainstorming on a new project, I stumbled upon a problem. I would like to create a neural network to encode a sequence of objects and get the gist of the sequence. However, the sequence is of ...
0
votes
0answers
25 views

What is the purpose of grids in YOLO?

Considering the YOLO algorithm. Assume: Input image is n x n x 3 Number of anchor boxes is m For each anchor box, we have 1 (pc = probability of object) + 4 (4 variables to predict the bounding ...
2
votes
0answers
36 views

Terminology question: distinguishing two meanings of “loss function”?

I've heard people use "loss function" to refer to 2 different things: 1) A real-valued function of a label, $y$, and a prediction $\hat{y}$. 2) A real-valued function of a parameter $\theta$; this ...
0
votes
1answer
180 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
0answers
16 views

Training a multi-layer perceptron (MLP) with a modified basis function

Consider a simple 3-layer MLP such as this. Each hidden layer implements y=xw+b where y is the output activation matrix of the ...
3
votes
1answer
1k views

From Markov Decision Process (MDP) to Semi-MDP: What is it in a nutshell?

Markov Decision Process (MDP) is a mathematical formulation of decision making. An agent is the decision maker. In the reinforcement learning framework, he is the learner or the decision maker. We ...
0
votes
0answers
4 views

Using supply as feature in price predictor model

On a machine learning model that outputs the optimum price of a product (ex: cars listed on some website), would it make sense to use the number of instances of that product as a feature? In the case ...
2
votes
1answer
217 views

A proof of within-cluster sum of squares?

Anyone can provide a proof of the following equation as in @cardinal 's answer? $x_i$ and $x_j$ are vectors from the same clusters。 $\sum_{i,j} ||x_i - x_j||^2 = \sum_{i \neq j} ||(x_i - \bar{x}) - (...
1
vote
1answer
1k views

Interpret learning curves: Training error and validation error are low

I am confused about how to evaluate this result. From this link, it seems like my model is just right, I just want to make sure that my result is a good fit. Any helps would be appreciated. Thanks! ...
3
votes
1answer
234 views

Work flow for multinomial text extraction of OCR data

I am interested in recreating (for cost and data sharing discrepancies) a solution that machine learning solution companies like ABBYY have done. I would like to be able to take purchase order PDFs ...
0
votes
0answers
32 views

Why using variational inference to do minimization?

I'm reading a paper on conditoinal random fields. They arrive at a formulation for energy, and they go like this: "minimizing this is intractable" What does that mean? I heard about intractable ...
0
votes
4answers
481 views

Kernel Selection [closed]

First, I want to know how to analyze a dataset to discover its pattern. And second, how can I select the best kernel function for classifying a dataset?
0
votes
1answer
365 views

Why does co-occurrence matrix have 0 values in diagonal?

I am studying this document but I can't understand why in the word-word co occurrence matrix the terms in the diagonal are all 0. Or in other words why I do not count the word itself in the co-...
0
votes
0answers
23 views

What Does $y=f_W(X)=\overline{\overline{X}W} \vec{1}$ Mean?

I am reading this paper and I came across this formula: $$y=f_W(X)=\overline{\overline{X}W} \vec{1}$$ I understand that $$X$$ is an input entity and $$W$$ is the binary matrix. What I am having ...
0
votes
0answers
10 views

Could someone use this concrete data set to illustrate how to compute the average Gain?

Chapter 3, Page 86 of Tom M. Mitchell. Machine Learning (free) says One practical issue that arises in using GainRatio in place of Gain to select attributes is that the denominator can be zero or ...
1
vote
1answer
266 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
0
votes
0answers
18 views

Analysing arrays of image data with Machine Learning Models

I am trying to do Machine Learning on arrays or vectors describing images. The target variable is a category I am trying to predict. I have multiple features that each contain arrays describing the ...
0
votes
0answers
13 views

Extracting metrics from natural language

Imagine a text like The revenue amounted to mEUR 124 during the year while last year it resulted in mEUR 100. I want to use ML methods to extract two outputs like ...
1
vote
0answers
13 views

Reference: Data-Dependent Early Stopping Criterion for Deep Learning?

In the context of non-parametric regression, this paper provides an data-dependent rule for optimal early stopping, when learning an unknown function $f^{\star}$ lying in some RKHS. Here, one stops ...
-5
votes
0answers
35 views

Why NN works better SVM? [closed]

is Neural Network Ensemble give best prediction over other prediction models or algorithms? If yes, what type of Neural Network?
0
votes
0answers
17 views

Derivation of Bayes classifier in Murphy's book

I am reading Kevin Murphy's Machine Learning book (MLAPP) and want to know how he got the expression for the Bayes classifier using minimization of the posterior expected loss. He wrote that the ...
0
votes
0answers
7 views

What is best practice when standardizing a truncated numeric variable with lots of zeroes?

What is best practice when standardizing truncated numeric variables with lots of zeroes (like 80% of the obs.)? To provide an example, I have a variable counting number of days per year several ...
0
votes
1answer
39 views

Using data from multiple sources with same features in classification problem

Suppose that I am doing a classification problem where I classify people into two categories as bullied or not. In such type of ...
1
vote
0answers
51 views
+50

Choosing learning rate with 2nd order method - minimizing parabola in one step?

In parabola $(\theta,g)$ values are in line $(g=f'(\theta))$ - we can get slope of this line e.g. by dividing their standard deviations: $$ \mu = \frac{\sigma_\theta}{\sigma_g}=\sqrt{\frac{var(\...
1
vote
0answers
17 views

Compact/Vectorized Multiclass Logistic Regression Hessian

I know that the Hessian of the categorical cross entropy w.r.t the weights is given by $$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$ ...
1
vote
1answer
156 views

what is the method in dictionary learning which does not have a overcomplete dictionary?

what is the method in dictionary learning which does not have a overcomplete dictionary? and what is the difference in minimization between these two methods (one using overcompelte dictionary and ...
0
votes
1answer
26 views

Why do we penalize individual example divergence in variational autoencoder?

In variational autoencoder, we want to learn a mapping between input space $X$ and latent space $Z$, and $z\in Z$ is related to $x\in X$ with $z\sim MVN(\mu(x), \Sigma(x))$. In addition, we desire ...
0
votes
0answers
12 views

Compute gradient after bitwise OR [closed]

I have to calculate the Intersection over Union (IoU) of two segmentations. For that, at some point I have to calculate the bitwise OR of two tensors. I am doing this with ...
0
votes
0answers
32 views

Imbalanced data classification with GLM giving very poor results [duplicate]

I have a loan defaulters dataset and it is highly imbalanced as shown below: 0 1 33108 673 I have tried SMOTE to balance the dataset, as shown below: ...
0
votes
1answer
157 views

machine learning with asymmetric training data frequencies

I have about a million rows of data being collected every day, and I am trying to predict a government figure which is released on a less frequent basis, about once a month. Compared to a traditional ...
2
votes
1answer
221 views

How do I implement missing value patterns?

I have a training data set and I was able to find some interesting patterns in the missing values, and I used binary variables in order to represent the missingness. I am going to train a model, say a ...
0
votes
0answers
14 views

Finding Overlapping Topic Clusters in Labeled Text Data

I have a dataset which consists of two heavily interconnected classes. One is about "Cognitive Linguistics", and the other is "Not Cognitive Linguistics", both are under the umbrella term "Linguistics"...
0
votes
1answer
28 views

Can I apply PCA on continuous data and reduce the dimensions and keep categorical data as it is?

I have a dataset which contains 95 highly correlated continuous variables and other 3 categorical variables. I want to reduce the dimension of the data and by that I can deal with correlation as well. ...
0
votes
2answers
2k views

Learning image embeddings using VGG and Word2Vec

Background: In word2vec we pass in a one-hot encoding of our target word into a simple neural network which is trained to predict context words from a window around our target. We eventually take the ...
0
votes
0answers
5 views

Signal Embeddings using the skip-gram or CBOW model

So my work involves looking at a bunch of waveforms in the context of classifying events. I often am looking for new ways to represent my waveforms, and in my searching, I came across audio embeddings ...
1
vote
1answer
34 views

How to evaluate whether model is overfitting or underfitting when using cross_val_score and GridSearchCV?

This is something that has been written about extensively, but I'm just confused about a couple of particular things which I haven't found a clear explanation of. When cross validation is not used, ...
45
votes
6answers
25k views

Why is multicollinearity not checked in modern statistics/machine learning

In traditional statistics, while building a model, we check for multicollinearity using methods such as estimates of the variance inflation factor (VIF), but in machine learning, we instead use ...