Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

4
votes
1answer
87 views

What are multi-variable calculus pre-requisite for Machine Learning

I wanted to complete calculus pre-requisites for machine learning class. I am doing an online course of multi-variable calculus. Can someone please suggest what lectures after Lecture 15 are relevant. ...
5
votes
5answers
130 views

What is the best way to detect repetition in xyz data for purposes of splitting data?

I'll use this picture to explain What I want to do is define some patterns as trained patterns. Then given data I want to be able to determine if the pattern exists in the dataset, and if it does ...
3
votes
0answers
45 views

Perceptron trained on time series always predicting the same answer

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
1
vote
0answers
19 views

Gradient descent vs Contrastive Divergence

What are the differences (if any) between gradient descent and Contrastive Divergence? I understand how gradient descent is used to train neural networks via back-propogation, but I've just started ...
3
votes
0answers
35 views

K-fold cross-validation for time series with dynamic target variable (Scikit)

I would like to do a K-fold cross-validation on time series data (market data) with a two class classification target. My test folds must be forward looking and of a fixed size ...
0
votes
1answer
25 views

How do I add more training data to pre-trained deep learning net

Am kind of new to deep learning. I have however trained a number of face images and achieved relatively good recognition rate. Now my question is if my client will be taking a few more pictures daily ...
2
votes
1answer
36 views

SVM training and testing error interpretation

I am new to machine learning and I just used SVM for the first time to analyze my dataset... Now I have created a figure that displays the training and testing error of the model as a function of ...
1
vote
1answer
33 views

Which are the suitable classification algorithms when the number of categories are more than 1000

Combination of categories is not possible as each class is a distinct brand. One similar challenge was classifying objects using images (link below) but I could get any specific direction from few ...
0
votes
0answers
4 views

Hard partitioning of the association matrix

I obtain a co-association matrix $n \times n$ that corresponds to the maximum likelihood estimate of the probability of pairs of variables being in the same cluster. Further suppose that there are $k$ ...
2
votes
0answers
54 views

machine learning techniques for longitudinal data

I was wondering if there were any machine learning techniques (unsupervised) for modelling longitudinal data? I've always used mixed effects models (mostly non-linear) but I was wondering if there are ...
0
votes
1answer
23 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive ...
2
votes
1answer
30 views

What is the definition of a kernel on vertices or edges?

I am currently trying to perform clustering on a collection C of undirected and unlabeled graphs. I decided to use to a kernel on graphs to obtain the kernel matrix of C. Then I can derive the ...
0
votes
0answers
15 views

How to pre-process data & tune hyperparameters of Doc2Vec?

I am using doc2vec for getting document similarity (unsupervised learning). I read that we need to shuffle the input matrix to doc2vec & reduce the learning rate for better performance. But the ...
4
votes
1answer
101 views

Why do we want an objective function to be a convex function?

I understand that a convex function is a great object function since a local minimum is the global minimum. However, there are non-convex functions that also carry this property. For example, this ...
1
vote
1answer
39 views

Finding best weights for ranking

I have a problem concerning Data Science and Machine Learning, and maybe somebody could share a hint on how to accomplish or where to begin with. Thanks in advance. The thing is I have an application ...
1
vote
0answers
31 views

How does batch normalization compute the population statistics after training?

I was reading the batch normalization (BN) paper (1) and it said: For this, once the network has been trained, we use the normalization $$\hat{x} = \frac{x - E[x]}{ \sqrt{Var[x] + \epsilon}}$$ ...
1
vote
1answer
54 views

How and why does Batch Normalization use moving averages to track the accuracy of the model as it trains?

I was reading the batch normalization (BN) paper (1) and didn't understand the need to use moving averages to track the accuracy of the model and even if I accepted that it was the right thing to do, ...
1
vote
0answers
15 views

Reference request about feature maps in ML

Can someone kindly link to some recent papers on understanding feature maps in ML? It would help to get an idea of what are the recent issues there that people have been working on with regards to ...
1
vote
0answers
24 views

From MDP to SMDP: What is it in a nutshell

Markov Decision Process (MDP) is a mathematical formulation of decision making. An agent is the decision maker. In the reinforcement learning framework, he is the learner or the decision maker. We ...
0
votes
0answers
54 views

Free space detection on image

I have set of images with front or back rear cars which was obtained by cascade detector. I need to detect free space on image, (more precisely I need just to know how big car is in pixels). ...
0
votes
0answers
31 views

Plot a subtree from a big decision tree

I am working on my thesis using decision trees. I am presenting the resulting tree to show how they help in exploring data. My issue is that since the tree is big, I want to break it down into parts, ...
19
votes
5answers
1k views

Overfitting: No silver bullet?

My understanding is that even when following proper cross validation and model selection procedures, overfitting will happen if one searches for a model hard enough, unless one imposes restrictions on ...
4
votes
1answer
106 views

How can I experiment with Lagrange multiplier in PCA optimization?

Suppose we want to solve following optimization problem (it is a PCA problem in this post) $$ \underset{\mathbf w}{\text{maximize}}~~ \mathbf w^\top \mathbf{Cw} \\ \text{s.t.}~~~~~~ \mathbf w^\top \...
0
votes
1answer
24 views

Perfect recall and spurious states in hopfield networks

In Hopfield networks, one can apparently load perfect recall into the network (by having enough neurons compared to patterns). (Source) However, at the same time, it appears that spurious states (i.e....
1
vote
2answers
24 views

The importance of lambda in a regularization function, with respect to the hypothesis.

I'm working through some parts of Russel & Norvig's Artificial Intelligence, and this is the cost function they give: Cost(h) = EmpericalLoss(h) + λComplexity(h) How does choosing a value for ...
1
vote
0answers
26 views

Re-estimation of Hidden Markov Model Parameters without Knowing Hidden States

I have been working on Hidden Markov Models (HMM) for a while. I thought that I understood the basics of HMM, however, recently I have confused about a point. Here is the issue: Recall the 3rd ...
7
votes
2answers
215 views
+50

How to prove this Gaussian Mixture inequality? (Fitting/Overfitting)

Let f[x] be a Gaussian Mixture pdf with n terms of uniform weight, means $\{\mu_{1},...,\mu_{n}\}$, and corresponding variances $\{\sigma_{1},...,\sigma_{n}\} $: $$f(x)\equiv\frac{1}{n}\sum_{i=1}^{n}\...
0
votes
0answers
13 views

What does mean by the number of pixel positions in CNN

I am doing project in machine learning using deep CNN. I need to understand how to choose hyperparameters (number of filter, shape of filter, max pooling shaep,..). I am using image of 42*42 (...
0
votes
1answer
50 views

How to create a regression model if data points are structure as seen in the graph?

This is a graph of revenues for different products with the Y-axis showing normalized revenues (mean of 3 and SD of 1) and X-axis is weeks. I need do a regression analysis of sorts on this data and am ...
2
votes
0answers
20 views

What preprocessing techniques work well for autoencoding audio?

I am wondering what preprocessing techniques work well for autoencoding audio data? Specifically I have a dataset of ~0.5 second audio samples of people pronouncing digits 0-9 (think an audio version ...
1
vote
0answers
15 views

About derivation of “regression minimizes empirical risk”

I've been reading Vapnik's Statistical Learning Theory. In his book, regression is defined as $r(x) = \int ydF(y|x)$, where $F(y|x)$ is conditional (cumulative) probability distribution. The risk ...
2
votes
2answers
30 views

How do you know that the assumptions of the model have been satisfied, and it’s ok to run the algorithm? [closed]

when using any simple algorithm like logistic regression, svm or even complex ones, how could you know that it’s ok to run the algorithm and use it in the industry? for example :when using logistic ...
1
vote
3answers
27 views

Principal component analysis result

I'm trying to understand the result of PCA, thought you can help me to understand better. ...
0
votes
1answer
132 views

Gradient for logistic loss function

I would ask a question related to this one. I found an example of writing custom loss function for xgboost here: ...
1
vote
1answer
33 views

Feedforward Vs. Backpropagation Neural Network

I was taking this course Machine Learning-Coursera (Standford) by Andrew Ng In Week 4 and Week 5 we have given ...
1
vote
0answers
25 views

How to deal with related inputs?

I am a neural network newbie, so please don't blame me if the question is silly. How to handle related inputs in a neural network? Let's explain it with an example: Suppose my NN expects a vector of ...
1
vote
1answer
32 views

What is the proper name for “unknown data” set in machine learning?

As far as I know in practice the whole training set is usually split into training, validation and test[1] sets. Training set is used to train the model, validation to tune the parameters and test set ...
0
votes
0answers
14 views

Non-parametric smoothing a small sample time series with fixed/known t(0) and t(n)

I would be grateful for any suggestions on how to smooth a time series with the following properties: We observe $t(i)$ for every integer $i=0...T$. $0 < t(i) < \infty$ $T$ is typically small (...
2
votes
1answer
71 views

Find out if using k-fold cross-validation helped to overcome overfitting (Machine Learning standard)

One of the main way to overcome overfitting is using $K$ fold cross-validation, and as this paragraph in cross-validation wiki page says: The goal of cross-validation is to estimate the expected ...
2
votes
0answers
52 views

Calculating conditional probability in Bernoulli mixture model

I'm following the book Pattern recognition and machine learning by Bishop on Bernoulli mixture model, and trying to code it. But I don't understand how to calculate the conditional probability (page ...
0
votes
1answer
25 views

Neural Networks with Input 0

Imagine you have a Neural Network with Sigmoids. It has an input $x$ and so a node would output $\tanh x$ to a connection. The connection would then output $w*tanhx$ where $w$ is the weight of the ...
1
vote
0answers
6 views

How much variance is captured by the RFF maps?

The RFF maps here are possibly the most used feature maps. I was wondering if there are cases where anyone has theoretically estimated the total variance captured by these maps? Is any simplification ...
0
votes
0answers
16 views

is the time it takes for a network to perform will on a classification problem proportional to its learning rate?

I've followed a brief tutorial found here: http://outlace.com/Beginner-Tutorial-Theano/ I wanted to know what happens when I change the learning rate. At the rate it is set in the tutorial, ...
2
votes
1answer
27 views

Is it possible to add up the accuracy rates of 2 predictors?

Weather channel 1 has a 65% accuracy rate of predicting tomorrows weather Weather channel 2 has a 59% accuracy rate of predicting tomorrows weather Is it somehow possible to take into account ...
-1
votes
1answer
21 views

Is the EM-algorithm the same thing that variational inference in LDA?

I am new in the probabilistic topic modeling, and I need to understand deeply the LDA process, I understand what want to do the inference process in LDA, and I understand too that there is 2 "types" ...
2
votes
1answer
79 views

TOO low estimated SVM probability for most of the negative test examples?

I am using LIBSVM (as well as the fitcsvm and fitSVMPosterior of ...
1
vote
0answers
21 views

Deep Belief Network configuration for dice face recognition

I should develop a network that can read the result of throwing a dice. I have a dataset which consists on a synthetic collection of such images, together with the corresponding target values. Each ...
0
votes
0answers
37 views

how to understand 'end to end' in deep learning?

Recently, I do some literature research about CNN and find there is a concept of end to end training Such as the abstract in Fully Convolutional Networks for Semantic Segmentation How to ...
3
votes
0answers
32 views

Is it ok to use symmetric loss function when evaluation metric is asymmetric?

I completely understand that it's ok to use a loss function different from the evaluation metric. For example, accuracy isn't computationally feasible to optimize directly since it's not ...