Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Methods and principles of building "computer systems that try to automatically improve with experience."

0
votes
1answer
6 views

Stacked shallow autoencoders vs. deep autoencoders

In LeCun et. all "Deep Learning", Chapter 14, page 506, I found the following statement: "A common strategy for training a deep autoencoder is to greedily pretrain the deep architecture by training a ...
0
votes
1answer
219 views

Methods for unsupervised subset selection on categorical data

I am new to this. I have a set of survey data with 18 questions (columns/features) with 165 observations. Responses are ternary: True, False, Don't Know. Each question has a correct response, which ...
0
votes
1answer
19 views

Problem with the word 'machine' in the definition of machine learning by Mitchell in the book “Machine Learning”

The definition : A computer program is said to learn from experience E with respect to some task T and performance measure P, if its performance at task T, as measured by P, improves with experience ...
0
votes
0answers
6 views

Extracting correlated data

Lets say I have a dataset like this image here https://imgur.com/oKXBX8q. The top figure is a histogram of the underlying data points which tend to be distributed vertically, horizontal, and at some ...
0
votes
0answers
10 views

Incorrect practice of data preprocessing

Suppose I have a dataset and split it into big_train and test set, as usual. Now if I split further the big_train set into small_train and validation set, suppose I use PCA, there are $2$ approaches: ...
0
votes
0answers
14 views

How to derive the maximum *a posteriori* estimate when the prior distribution is Normal $N(m,r^2)$?

I am learning probabilities and I need a guru to help with this problem: Assume $p(y | x) = N(ax,\ s^2)$, where all quantities are scalars, $a$ and $s$ are known constants, and the prior ...
0
votes
0answers
20 views

Mathematically, what are the drawbacks of R-squared in evaluation a regression model?

I kept seeing articles about the drawbacks of R-squared (and that's why we need to have adjusted R-squared). One drawback is that: "Every time you add a predictor to a model, the R-squared increases,...
2
votes
1answer
14 views

How to interpret chart generated by gbm.perf function?

I'm new to GBM.Can you help me to understand the interpretation of gbm.perf function? I used following code in R best.iter = gbm.perf(train, method="cv") & got ...
108
votes
8answers
39k views

Why is Newton's method not widely used in machine learning?

This is something that has been bugging me for a while, and I couldn't find any satisfactory answers online, so here goes: After reviewing a set of lectures on convex optimization, Newton's method ...
0
votes
0answers
6 views

How to find the distance confidence between the sum of random variables and a given target value?

Given a distribution X($E[X]$, $Var[X]$) and a target value T, I am wondering how to find a value N such that $|\sum_{i=1}^{i=N}X_i - T|$ is minimized. (i) For a large T, it makes sense to pick $N = ...
0
votes
1answer
16 views

Is the grid in a self organizing map static?

I'm trying to write my own SOM in python, and after reading material from several sources (and watching video tutorials) I think I understand all the steps. There is however one issue that I want to ...
2
votes
0answers
206 views

When is logistic regression minimizing under squared error loss the same as maximizing binomial likelihood? [duplicate]

Implementing logistic regression and getting different results depending on whether I minimize squared error or maximize log likelihood. When are the two equivalent?
2
votes
1answer
273 views

Early stopping by epoch-limit

Is limiting the maximum number of training epochs during optimization a standard regularization process? I have seen it in many source codes of matrix factorization implementations, but I was not ...
2
votes
1answer
21 views

agnostic PAC model: Learnability and Bias-Complexity Trade-off

I am reading "Understanding Machine Learning: From Theory to Algorithms." In Chapter 5.2, it says that choosing the hypothesis class $\mathcal{H}$ to be a very rich class decreases the approximation ...
1
vote
0answers
10 views

Machine learning for product names

I have a machine learning challenge I may be over thinking. I have a set of 3.5 million products (not unique, there are multiple instances of each product). Each product has a "description" from it's ...
0
votes
0answers
11 views

variational lower bound confusion

In this blog describing variational inference under the section KL divergence and ELBO they mention that in the equation $$p(x) = \frac{w(x)}{Z}$$ we can substitute $w$ and $Z$ with: $$Z = p(x;\...
0
votes
0answers
8 views

Loss function for a risk neutral binary classification

For binary classification task, with samples labeled $y=0$ and $y=1$, a neural network has one output node with sigmoid activation function, producing predictions $\hat{y}\in(0;1)$. Is the following ...
2
votes
0answers
33 views

compare Bayesian linear regression vs standard linear regression

1st question, I recently learnt bayesian linear regression, but I'm confused that in what situation we should use bayesian linear regression, and when to use standard linear regression? What is the ...
0
votes
0answers
20 views

How to explain why KNN has good classification performance (have example)

I have a good prediction results on KNN-DTW group classification (group labels are either 0 or 1). But I don't know how to explain how discriminative the 2 groups are. Then I tried k-means and tsne ...
3
votes
1answer
89 views

Neural Networks - Strategies for problems with high Bayes error rate

I am building a Neural Network for a binary classification problem where the Bayes error (lowest possible error rate) is probably close to 50%. What makes the task easier is that I don't need to make ...
36
votes
5answers
13k views

LDA vs word2vec

I am trying to understand what is similarity between Latent Dirichlet Allocation and word2vec for calculating word similarity. As I understand, LDA maps words to a vector of probabilities of latent ...
1
vote
1answer
37 views

sklearn Support Vector Regression - test data prediction is constant

I am just getting into learning some basic machine learning for a project at university and I am having a little trouble with SVR on sklearn. When training a model I can change the epsilon value and ...
0
votes
1answer
24 views

Batch processing in a neural network

I am trying to understand how each batch is processed in a neural network. I understand that if we have a training set $X=\{x_1,...,x_{|X|}\}$ and we specify a batch size of $n$ than the neural ...
0
votes
1answer
13 views

Retuning hyperparameters of the baseline when comparing it with a new model

I have a baseline model which has certain hyperparameters to tune (it's actually a neural network, but I don't know if it's important in this context). I want to compare it with my own extension of ...
2
votes
1answer
82 views

bias variance tradeoff — properties that do not follow

Going through this lecture note on bias-variance trade-off, I didn't follow the latter part of this paragraph. It shows the common situation in practice that (1) for simple models, the bias ...
0
votes
0answers
34 views

What is the difference between KNN and K* algorithms? [on hold]

Are KNN and K* under the same category of machine learning algorithm? The explanation on KStar can be found on: http://weka.sourceforge.net/doc.dev/weka/classifiers/lazy/KStar.html Whereas KNN is a ...
2
votes
0answers
72 views

Classification: how important is the sample-to-feature ratio?

Some people mention you should have at least 5 times as many samples as features for classification problems 1. I've also heard people on here saying the sample-to-feature ratio is arbitrary and ...
3
votes
3answers
1k views

Deriving REINFORCE algorithm from policy gradient theorem for the episodic case

In the draft for Sutton's latest RL book, page 270, he derives the REINFORCE algorithm from the policy gradient theorem. The first part is the equivalence $$\sum_{s}d_\pi(s)\sum_{a}{q_\pi(s,a)\nabla\...
1
vote
1answer
246 views

Error Backpropagation, Christopher Bishop “Pattern Recognition and Machine Learning”

I'm trying to understand the description of the error backpropagation algorithm as explained in Christopher Bishop's book, in particular, section 5.3.1 "Evaluation of error-function derivative". The ...
0
votes
1answer
21 views

High AUC and Accuracy but weird output in confusion matrix

I am working on image classification problem to determine gender given a face. The dataset is located here gender face dataset on kaggle (link to my notebook). The class distribution is as follows. <...
0
votes
1answer
226 views

How to build classification model towards some rare response classes?

I was asked to build a predictive classification model that can predict some types of response. I am interested in 6 classes, however, the total occurence of these 6 classes (out of almost half a ...
1
vote
1answer
26 views

Feedforward networks: methods to avoid two neurons in the same layer learning the same weights and biases?

There are many questions on this site which have to do with "what happens when two neurons have the same weights/biases" and I am not asking about that. However, it is occasionally the case that a ...
0
votes
0answers
19 views

PySpark ML: What to do when a logistic regression model is not generalizing?

I created a logistic regression model using PySpark ML. My feature set consists of both categorical and continuous features, and I ran the following to pre-process them: Categorical features: All of ...
0
votes
1answer
254 views

SVM classifier convergence

I was implementing a SVM Classifier using scikit library on a MNIST dataset available on Kaggle. Everything was going well until one of my friend asked me a ...
0
votes
1answer
53 views

How to normalize data of 0s and 1s?

the data consists of one 2, so it's not binomial. is there any way to transform it to fit the normal assumptions? I tried square root and log, both didn't work
1
vote
0answers
8 views

Learning useful semantic representations of data

Training a neural network on its final task (e.g. classification) right from the beginning is not always the best way to go. I'd like to make a short list of recognized methods of motivating a NN to ...
0
votes
1answer
43 views

Are the same number of trees required to compare Random Forest against GBM?

My training set has 13,737 observations with 53 predictors. I need to compare the accuracy of Random Forest vs. GBM. For Random Forest, I set ntree = 128 [based on ...
2
votes
2answers
5k views

Multivariate statistics vs machine learning?

Are multivariate statistics and machine learning solving the same problems? I saw that their books are about the same topics, so I have the impression that they are solving the same problems and ...
0
votes
0answers
12 views

Is there any formal explanation for the sensitivity of AdaBoost to outliers?

AdaBoost is known to be sensitive to outliers & noise. However, the explanation seems to be hard to found or nontrivial.
2
votes
1answer
434 views

Why do we need the score function in reinforcement learning?

I have a hard time grasping the need for policy optimization and say the log kernel trick/score function. Instead of using the score function, why do you not simply optimize for the highest reward and ...
0
votes
0answers
11 views

Pointwise Mutual Information Word Dependency

I have pre-defined concepts which are either a single word or couple of words that refer to a concept.( In the context of machine learning for instance, covariance matrix is a concept). I am trying to ...
5
votes
2answers
660 views

Why there is square in MSE (mean squared error)?

Please forgive me for such a beginner question, since I'm learning stats . & machine learning. I'm trying to understand Mean Squared Error. I understand the "Mean Error", the Mean of Errors ...
2
votes
2answers
480 views

Why uppercase for $X$ and lowercase for $y$?

Why is it most of the time (in many websites, articles or demonstration) the feature variable (columns) is denoted by a upper-case 'X' whereas the target variable is a lower-case 'y'? Looks more like ...
0
votes
0answers
7 views

String similarity evaluation

I have been trying some String similarity functions such as Jaro Winkler, Cosine and a rule based one. I want to evaluate their performance. It is easy to get the True Positives and False ...
0
votes
1answer
22 views

Why does different factorisation matter in Markov networks?

I have been reading about Markov Networks that given some set of factors we can construct a unique graph G but not the other way around: "It should also be noticed that, given a set of factors, the ...
0
votes
0answers
40 views

Which machine learning technique suits this use case?

I have two different zones say for ex. IN and OUT and each zone with two features i.e., A & B. Now I have a target with the same two features and with the help of this, I need to identify, to ...
2
votes
1answer
71 views
1
vote
0answers
13 views

Expected Classification error using indicator functions

I am given three variables (latter two are binary) $A,B$, and $C$, where $A$ = input vector, $B$ = whether data was chosen, and $C$ is the true label. $A$ and $B$ are conditionally independent given $...
4
votes
1answer
36 views

What does it mean to freeze or unfreeze a model?

I'm going over the fast.ai course on deep learning now, and there's frequent calls to freeze/unfreeze methods in the lesson ...
3
votes
0answers
27 views

PCA, SMOTE and cross validation - how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...