Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

Non-GLM Models for Fractional Response Variable

I am searching for a machine learning algorithm that I can use to predict customer retention/churn rate. My response variable is a proportion in the range 0 to 1 (0 and 1 inclusive). I am using R. The ...
1
vote
1answer
55 views

Number of neurons in the output layer

In a classification problem, how do you decide on the number of output neurons you have in your neural network? Is the number of neurons equal to the number of classes you have? Is there a limit on ...
15
votes
5answers
380 views

On the importance of the i.i.d. assumption in statistical learning

In statistical learning, implicitly or explicitly, one always assumes that the training set $\mathcal{D} = \{ \bf {X}, \bf{y} \}$ is composed of $N$ input/response tuples $({\bf{X}}_i,y_i)$ that are ...
0
votes
1answer
75 views

Cross-validation of multiple subjects with multiple instances

I have a training set of 50 subjects with about 550-600 measurements each. One measurement consists of 24 features and one class label (1 or 0). So my data looks like this (simplified): ...
0
votes
0answers
13 views

Can Lloyd's algorithm be implemented in Knn?

I have seen papers that uses Lloyd's algorithm that could optimize K-Means, but i was wondering if the algorithm can be used to K nearest neighbor.
3
votes
1answer
71 views

Learning functional analysis for studying kernels

I'm trying to learn more about kernel machine theory and I've discovered that I need to learn a lot of background math, and so I'm looking for some good resources for this. In particular: I've got ...
0
votes
1answer
67 views

SVM model training set vs test set

I am trying to train an SVM model using Forest Fire data. I split up my data into a test and training set. I am fairly new to this type of analysis but I'm not sure what role the test data plays or ...
0
votes
1answer
57 views

Why is optimisation solved with gradient descent rather than with an analytical solution? [duplicate]

I'm trying to understand why, when trying to minimise an objective function, gradient descent is often used, rather than setting the gradient of the error to zero, and solving it analytically. In ...
0
votes
0answers
37 views

Cross-validation in combination with Multiple Imputation

I am working on a project where I want to cross-validate a Machine Learning algorithm (not logistic regression) on multiply imputed data. My question is, how can I use the training data to multiply ...
0
votes
2answers
17 views

How to normalize feature vectors for concatenating

I have two different feature vectors of completely different scale, which are to be used as training data for machine learning algorithm. When I concatenate them, should I scale and normalize them ...
0
votes
1answer
30 views

How to deal with Features having high cardinality

I am building a simple classification model, the classification model is used for identifying negative or positive feedback on a ticketing system. The tickets have alot of categorical data with high ...
1
vote
1answer
36 views

References for learning about online random forests

I am new to concepts of random forest. Can someone provide relevant sites where I could get learn more about using random forests to learn incoming data like an online algorithm?
0
votes
2answers
49 views

Neural Network with unknown number of Neurons in output layer

Is it possible to design a network with an unknown number of neurons in the output layer? I am trying to solve a classification problem, where I use motorcycles' exterior color, interior color, and ...
0
votes
0answers
23 views
0
votes
0answers
22 views

How to predict on part of image after training on other part of image?

I have images of identity cards (manually taken so not of same size) and I need to extract the text in it. I used tesseract to predict bounding boxes for each letter and am successful to some extent ...
0
votes
0answers
15 views

optimizing 2 factors - Can you use NSGA-II to optimize this?

I have 3 machines A, B and C. I would like to rank the machines based on which machine maximizes Score1 and Score2. Score1 and Score2 are performance measures that rang from 0-100%. Below is some ...
1
vote
1answer
12 views

Simple way to combine predictions from multiple classifiers?

I have predictions from 3 binary classifiers (SVM, RF and NN) and would like to combine them in some way. I'm aware of the notion of ensemble learning methods but I was wondering if it would be valid ...
0
votes
1answer
59 views

Why would LASSO not shrink irrelevant features to zero?

Assume I have 10 features to predict an outcome and I use LASSO regression. Let's say the RMSE of the test set is 20. Now, I introduce 5 more features and predict the same outcome, and I also use ...
1
vote
1answer
37 views

Predicting the maximum of a function given a set of samples

The main aspects of the question are highlighted in bold Let $f: \mathbb{R}^n \mapsto \mathbb{R}$ be a function. Supposing that we have access to a set of samples $(X,Y)$ obtained by sampling the ...
1
vote
2answers
34 views

How to achieve a nonlinear decision boundary?

What would be the architecture of the neural net that would produce the following nonlinear decision boundary? Will the hidden layer compute some nonlinear combinations of inputs? or it will create ...
0
votes
1answer
15 views

problems with sorting the cases and controls in machine learning

I am using weighted majority voting in machine learning. Will it create the problem if I train the data using all control data first followed by all case data? case = 1,control = 0 [0 0 0 0 0 1 1 1 1 ...
1
vote
0answers
6 views

About the evaluation of covariance of linear Gaussian model in PRML

Section 8.1.4 of Pattern Recognition and Machine Learning introduces the linear Gaussian model where each node has distribution $$ p(x_i \big|pa_i) = \mathcal{N}\left ( x_i \Bigg| \sum_{j\in ...
1
vote
0answers
15 views

Reference summarizing various machine learning algorithms' computational complexity

For example, suppose you train a linear regression model using the Normal Equation, on a training set $\mathbf{X}$ containing $m$ instances and $n$ features. The Normal Equation requires computing ...
1
vote
0answers
16 views

How to identify a linear additive regression equation?

I am looking for some material/articles that tell the difference between linear additive and linear non-additive equations. For example, based on my current understanding I can say that this ...
0
votes
0answers
30 views

Understanding recurrent SVM in volatility estimation of GARCH model

I read Chen et al. "Forecasting volatility with support vector machine-based GARCH model" (2010) where they implented a recurrent SVM procedure to estimate volatility by a GARCH based model. The ...
0
votes
1answer
23 views

General Criteria for Feature Quality

It is generally accepted that the most important factor for successful machine learning is quality feature engineering: Feature Engineering is the Key At the end of the day, some machine ...
0
votes
0answers
7 views

What are some offline metrics for sparse data set

I have a real world machine learning problem: Predicting whether user will buy a item on our website. The model we used is point wise logistic regression and the offline metric is AUC. With about ...
3
votes
1answer
113 views

Why do deep learning practitioners forego PCA for ZCA?

I have an understanding of PCA and ZCA, read a similar question on the subject which, unfortunately, does not have the specific answer to my question. I understand the benefits of data whitening: ...
0
votes
0answers
7 views

How to model Dynamic Bayesian Network for represent mixture of gaussian hidden markov model

I have Mixture of Gaussian Hidden Markov Model of some process and i found some research represent Mixture of Gaussian Hidden Markov Model by Dynamic Bayesian Network and they told this can increasing ...
2
votes
1answer
130 views

PyMC3; create simple Linear Regression model with real-world datasets

The Linear Model I understand the concepts of Bayesian Inference in that the observed data, $x$, is fixed, and the parameters, $\theta$, are the random variables that follow a particular ...
1
vote
1answer
21 views

Trees of ensembles.

I have a large dataset (100k+), and it's growing everyday. I want to train it to predict a value (a regression problem). I've been finding that ensemble trees work the best for now, but in the ...
2
votes
1answer
89 views

Using von Mises-Fisher distributions for geo-spatial machine learning

There's an interesting paper about predicting the geographical co-ordinates of Twitter users based on the kinds of words that they use in their posts. I'd like to do something similar that involves ...
0
votes
0answers
20 views

Why is the reconstruction error for my training set larger than my test error using PCA on the MNIST data set? [duplicate]

I have a very strange behavior where I am trying to run PCA on the MNIST data set and then I check the test and train error. However, it seems that I get that the test error is lower than my train ...
0
votes
0answers
9 views

How to utilize the features available in the train data but not in the test data to train the algorithm?

I have recently worked on some kaggle competitions and found some of them provided some features only for the training data but not for the test. For example this Expedia one does not have session ...
0
votes
1answer
70 views

How would you represent this one-vs-all SVM accuracy?

I have a set on one-vs-all SVMs. Let's say I have three classes. I want to show FAR and FRR from the system, but I appear to get getting very large FRR values and very little FAR values. This is ...
0
votes
0answers
32 views

Principal Component Analysis intuition

I am trying to understand the basics of PCA. I am trying to figure out how this helps us reduce dimensionality. From what I understand, we have a set of data in N-Dimensional Space We find N ...
1
vote
1answer
32 views

Is the matrix coeff from MATLAB's pca the same as the left singular vectors of the centered data?

Consider the SVD of a centered data matrix: $$ X_{centered} = U \Sigma V^T$$ where a column of $X_{centered}$ is: $$ X_{centered} = x^{(i)} - \frac{1}{N} \sum^N_{n=1} x^{(n)} $$ is the matrix $ U ...
0
votes
0answers
19 views

Distributed PCA

I have a large data set. Large means many instances (~50000000) and many features (~25000). I will call my data matrix X where the rows are instances and columns are features. Lets say the number of ...
0
votes
0answers
10 views

Neural networks SVRG

Why aren't neural networks trained using SVRG(Stochastic Variance Reduced Gradient)? I've searched over internet and haven't found anyone who does so. Is that due to need to compute gradient twice ...
0
votes
0answers
26 views

How to describe my problem when my features are vectors?

My problem is a multivariate time series of measurements from a chemical sensor. There are $n$ different experiments made with as many different substances. Each experiment ranges over $t$ time steps. ...
3
votes
4answers
191 views

Why is gradient descent required?

When we can differentiate the cost function and find parameters by solving equations obtained through partial differentiation with respect to every parameter and find out where the cost function is ...
5
votes
2answers
139 views

Simple example of how “Bayesian Model Averaging” actually works

I'm trying to follow this tutorial on Bayesian Model Averaging by putting it in context of machine-learning and the notations that it generally uses (i.e.): ...
0
votes
0answers
37 views

Probability theory and machine learning questions?

I'm reading Kevin Murphy's book Machine Learning: A probabilistic Perspective and while I was reading I ran into few problems like in this Image, in generative classifier, how do we get the right side ...
0
votes
0answers
21 views

How does the fraction of retained PCA variance affect the accuracy of a model?

I was checking various tools for classification and optimisation; I trained a sample dataset using KNN. I got 100% accuracy with 95% PCA explained variance and 99.2% accuracy with 5% PCA explained ...
0
votes
1answer
26 views

type of image to create a dataset for image recognition using convolution neural network

I was trying to create a dataset for animal detection using convolution neural network. It was for some open source project. For the training and testing, I thought to create a dataset myself. for ...
0
votes
0answers
16 views

Binary classification with CNN for soccer ball detection doesn't converge

I'm working on a project where I want to detect classic soccer balls in live camera pictures using a Convolutional Neural Network. My Network is built up as follows: ...
0
votes
0answers
10 views

Can somebody describe what sparse models are in simple words?

I couldn't find any material that explains this. Is it also true that using Lasso regression will result in a sparse model only if there are any irrelevant variables/predictors?
3
votes
3answers
109 views

Prediction intervals for machine learning algorithms

I want to know if the process described below is valid/acceptable and any justification available. The idea: Supervised learning algorithms don't assume underlying structures/distributions about the ...
1
vote
0answers
18 views

What is the most efficient algorithm for online Non- negative Matrix Factorization (NMF)?

What is the most efficient algorithm for online Non- negative Matrix Factorization (NMF) in recent study? Thanks.
0
votes
0answers
36 views

Machine learning approach when facing low predictive power features

My dataset has 3.6k samples and 600+ one-hot encoded features. Each feature has between 5-2000 instances, averaging around 150. Intuitively, I don't believe that my features should have much ...