# Tagged Questions

Methods and principles of building "computer systems that automatically improve with experience."

87 views

### What are multi-variable calculus pre-requisite for Machine Learning

I wanted to complete calculus pre-requisites for machine learning class. I am doing an online course of multi-variable calculus. Can someone please suggest what lectures after Lecture 15 are relevant. ...
130 views

### What is the best way to detect repetition in xyz data for purposes of splitting data?

I'll use this picture to explain What I want to do is define some patterns as trained patterns. Then given data I want to be able to determine if the pattern exists in the dataset, and if it does ...
45 views

### Perceptron trained on time series always predicting the same answer

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
19 views

### Gradient descent vs Contrastive Divergence

What are the differences (if any) between gradient descent and Contrastive Divergence? I understand how gradient descent is used to train neural networks via back-propogation, but I've just started ...
35 views

### K-fold cross-validation for time series with dynamic target variable (Scikit)

I would like to do a K-fold cross-validation on time series data (market data) with a two class classification target. My test folds must be forward looking and of a fixed size ...
25 views

### How do I add more training data to pre-trained deep learning net

Am kind of new to deep learning. I have however trained a number of face images and achieved relatively good recognition rate. Now my question is if my client will be taking a few more pictures daily ...
36 views

### SVM training and testing error interpretation

I am new to machine learning and I just used SVM for the first time to analyze my dataset... Now I have created a figure that displays the training and testing error of the model as a function of ...
33 views

### Which are the suitable classification algorithms when the number of categories are more than 1000

Combination of categories is not possible as each class is a distinct brand. One similar challenge was classifying objects using images (link below) but I could get any specific direction from few ...
4 views

### Hard partitioning of the association matrix

I obtain a co-association matrix $n \times n$ that corresponds to the maximum likelihood estimate of the probability of pairs of variables being in the same cluster. Further suppose that there are $k$ ...
39 views

### How come “”mathematically“” that lasso shrinks the coefficient towards zero whereas ridge does not? [duplicate]

I understand them geometrically !
54 views

### machine learning techniques for longitudinal data

I was wondering if there were any machine learning techniques (unsupervised) for modelling longitudinal data? I've always used mixed effects models (mostly non-linear) but I was wondering if there are ...
23 views

### How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive ...
30 views

### What is the definition of a kernel on vertices or edges?

I am currently trying to perform clustering on a collection C of undirected and unlabeled graphs. I decided to use to a kernel on graphs to obtain the kernel matrix of C. Then I can derive the ...
15 views

### How to pre-process data & tune hyperparameters of Doc2Vec?

I am using doc2vec for getting document similarity (unsupervised learning). I read that we need to shuffle the input matrix to doc2vec & reduce the learning rate for better performance. But the ...
101 views

### Why do we want an objective function to be a convex function?

I understand that a convex function is a great object function since a local minimum is the global minimum. However, there are non-convex functions that also carry this property. For example, this ...
39 views

### Finding best weights for ranking

I have a problem concerning Data Science and Machine Learning, and maybe somebody could share a hint on how to accomplish or where to begin with. Thanks in advance. The thing is I have an application ...
31 views

### How does batch normalization compute the population statistics after training?

I was reading the batch normalization (BN) paper (1) and it said: For this, once the network has been trained, we use the normalization $$\hat{x} = \frac{x - E[x]}{ \sqrt{Var[x] + \epsilon}}$$ ...
54 views

### How and why does Batch Normalization use moving averages to track the accuracy of the model as it trains?

I was reading the batch normalization (BN) paper (1) and didn't understand the need to use moving averages to track the accuracy of the model and even if I accepted that it was the right thing to do, ...
15 views

### Reference request about feature maps in ML

Can someone kindly link to some recent papers on understanding feature maps in ML? It would help to get an idea of what are the recent issues there that people have been working on with regards to ...
24 views

### From MDP to SMDP: What is it in a nutshell

Markov Decision Process (MDP) is a mathematical formulation of decision making. An agent is the decision maker. In the reinforcement learning framework, he is the learner or the decision maker. We ...
54 views

### Free space detection on image

I have set of images with front or back rear cars which was obtained by cascade detector. I need to detect free space on image, (more precisely I need just to know how big car is in pixels). ...
31 views

### Plot a subtree from a big decision tree

I am working on my thesis using decision trees. I am presenting the resulting tree to show how they help in exploring data. My issue is that since the tree is big, I want to break it down into parts, ...
1k views

### Overfitting: No silver bullet?

My understanding is that even when following proper cross validation and model selection procedures, overfitting will happen if one searches for a model hard enough, unless one imposes restrictions on ...
106 views

13 views

### What does mean by the number of pixel positions in CNN

I am doing project in machine learning using deep CNN. I need to understand how to choose hyperparameters (number of filter, shape of filter, max pooling shaep,..). I am using image of 42*42 (...
50 views

### How to create a regression model if data points are structure as seen in the graph?

This is a graph of revenues for different products with the Y-axis showing normalized revenues (mean of 3 and SD of 1) and X-axis is weeks. I need do a regression analysis of sorts on this data and am ...
20 views

### What preprocessing techniques work well for autoencoding audio?

I am wondering what preprocessing techniques work well for autoencoding audio data? Specifically I have a dataset of ~0.5 second audio samples of people pronouncing digits 0-9 (think an audio version ...
15 views

### About derivation of “regression minimizes empirical risk”

I've been reading Vapnik's Statistical Learning Theory. In his book, regression is defined as $r(x) = \int ydF(y|x)$, where $F(y|x)$ is conditional (cumulative) probability distribution. The risk ...
30 views

### How do you know that the assumptions of the model have been satisfied, and it’s ok to run the algorithm? [closed]

when using any simple algorithm like logistic regression, svm or even complex ones, how could you know that it’s ok to run the algorithm and use it in the industry? for example :when using logistic ...
27 views

### Principal component analysis result

I'm trying to understand the result of PCA, thought you can help me to understand better. ...
132 views

### Gradient for logistic loss function

I would ask a question related to this one. I found an example of writing custom loss function for xgboost here: ...
33 views

### Feedforward Vs. Backpropagation Neural Network

I was taking this course Machine Learning-Coursera (Standford) by Andrew Ng In Week 4 and Week 5 we have given ...
25 views

### How to deal with related inputs?

I am a neural network newbie, so please don't blame me if the question is silly. How to handle related inputs in a neural network? Let's explain it with an example: Suppose my NN expects a vector of ...
32 views

### What is the proper name for “unknown data” set in machine learning?

As far as I know in practice the whole training set is usually split into training, validation and test[1] sets. Training set is used to train the model, validation to tune the parameters and test set ...
14 views

### Non-parametric smoothing a small sample time series with fixed/known t(0) and t(n)

I would be grateful for any suggestions on how to smooth a time series with the following properties: We observe $t(i)$ for every integer $i=0...T$. $0 < t(i) < \infty$ $T$ is typically small (...
71 views

### Find out if using k-fold cross-validation helped to overcome overfitting (Machine Learning standard)

One of the main way to overcome overfitting is using $K$ fold cross-validation, and as this paragraph in cross-validation wiki page says: The goal of cross-validation is to estimate the expected ...
52 views

### Calculating conditional probability in Bernoulli mixture model

I'm following the book Pattern recognition and machine learning by Bishop on Bernoulli mixture model, and trying to code it. But I don't understand how to calculate the conditional probability (page ...
25 views

### Neural Networks with Input 0

Imagine you have a Neural Network with Sigmoids. It has an input $x$ and so a node would output $\tanh x$ to a connection. The connection would then output $w*tanhx$ where $w$ is the weight of the ...
6 views

### How much variance is captured by the RFF maps?

The RFF maps here are possibly the most used feature maps. I was wondering if there are cases where anyone has theoretically estimated the total variance captured by these maps? Is any simplification ...
16 views

### is the time it takes for a network to perform will on a classification problem proportional to its learning rate?

I've followed a brief tutorial found here: http://outlace.com/Beginner-Tutorial-Theano/ I wanted to know what happens when I change the learning rate. At the rate it is set in the tutorial, ...
27 views

### Is it possible to add up the accuracy rates of 2 predictors?

Weather channel 1 has a 65% accuracy rate of predicting tomorrows weather Weather channel 2 has a 59% accuracy rate of predicting tomorrows weather Is it somehow possible to take into account ...
21 views

### Is the EM-algorithm the same thing that variational inference in LDA?

I am new in the probabilistic topic modeling, and I need to understand deeply the LDA process, I understand what want to do the inference process in LDA, and I understand too that there is 2 "types" ...
79 views

### TOO low estimated SVM probability for most of the negative test examples?

I am using LIBSVM (as well as the fitcsvm and fitSVMPosterior of ...
21 views

### Deep Belief Network configuration for dice face recognition

I should develop a network that can read the result of throwing a dice. I have a dataset which consists on a synthetic collection of such images, together with the corresponding target values. Each ...