Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

2
votes
1answer
50 views

How to derive the recursive equation for back propagation for neural networks for $\delta_j = \frac{\partial E_n}{ \partial a_j} $

I am following the derivation for back propagation presented in Bishop's book Pattern Recognition and Machine Learning and had some confusions in following the derivation presented in section 5.3.1. ...
0
votes
1answer
37 views

The derivation of $\delta_j = \frac{\partial E_n}{ \partial a_j}$ errors for hidden units in back propagation for neural networks with the chain rule

I was trying to understand the derivation for back propagation for multi-layer neural networks from Bishop's Pattern Recognition and Machine Learning book. Specifically I was reading section 5.3.1 ...
2
votes
1answer
46 views

How is convolutional network used to locate logos in images?

I have a large set of logos (think of it as kind of logos of automobile companies). Now, I want to train a convolutional network to locate the logo in a given image. Are there any papers that talk ...
0
votes
0answers
15 views

How can one use weighted majority voting when the “truth” is unknown?

I have a dataset with five binary "voters" and a binary response. Unfortunately, I do not know the "truth" for the response, but the assumption is that more votes is more likely to be true. I would ...
0
votes
1answer
35 views

Dependence of Error: Does it matter for data-driven models?

Linear regression assumes that the errors of the response variable are independent of each other. Lets assume that a data-driven model like a random forest or multi-layer perceptron is trained/formed ...
0
votes
0answers
26 views

The infamous cross device identification and how to go about it

I am trying to work on a cross device user identification problem wherein I have mobile data,that is apps visited and mobile attributes; as well as browser data via cookies, that is websites visited ...
6
votes
1answer
56 views

What does VC dimension tell us about deep learning?

In basic machine learning we are taught the following "rules of thumb": a) the size of your data should be at least 10 times the size of the VC dimension of your hypothesis set b) a neural network ...
1
vote
2answers
50 views

What does the notation $t_{nk}$ mean for neural networks in Bishop's Pattern Recognition book?

I was reading Bishop's Pattern Recognition book, specifically, I was reading his notation for expressing the error just before back propagation. The particular equation I am a little confused about is ...
1
vote
0answers
12 views

Classifier for small event frequencies [closed]

I have a classification task where I need to detect defects that occur at a very low frequency, i.e. parts per thousand. In addition, the nature and characteristics of defects is not well understood. ...
2
votes
0answers
34 views

Should I make decisions based on micro-averaged or macro-averaged evaluation measures?

I ran a 10-fold cross validation on different binary classification algortihms, with the same dataset, and recieved both Micro- and Macro averaged results. It should be mentioned that this was a ...
1
vote
1answer
29 views

optimal sequential sampling in gaussian process models

Let's say we have a one dimensional dataset of 24 points along with their responses. I am reserving three boundary points for testing (i=1,23,24) and i am fitting a Gaussian process model based on a ...
0
votes
1answer
87 views

Why does my neural net fail to learn higher frequency sine waves?

I am testing my neural network implementation. I have an input layer with a single unit, one hidden layer consisting of 65 tanh units, and an output layer ...
0
votes
0answers
17 views

combining two sources of knowledge about covariates in logistic regression model

i have a number of variables and a dataset and i want to build a linear regression model with shrinkage like lasso. i have also another information about my covariates on their relation with ...
0
votes
0answers
18 views

linear regression model with variables containing multiple attributes

consider we have explanatory variables with two attributes and we would like to create linear regression model. for example we have variable x1 which has two attributes x1.val1=2 and x1.val2=0.8 ...
0
votes
1answer
19 views

learning ranked instance similarity by machine learning

Here there are many vectors with rank. a = c(1, 2, 3, 5, 10,...) b = c(4,2,3,2,8,...) ... please note, here it's the rank of value but not the value itself in these vectors. There are a few ...
0
votes
0answers
10 views

How to get values on testdata in RSNNS

I have two files testi containing few numbers and testo containing their square roots. I have another test named file which contains some numbers for which I want their square roots. I used the ...
0
votes
1answer
23 views

What are the techniques used for learning in non-feedforward neural networks?

Suppose our network architecture has a hidden layer in which the hidden units are interconnected, then is there some sort of variation on backpropagation that is used? What about in general recurrent ...
5
votes
0answers
87 views

How to derive the pdf using cluster weighted and Kernel density estimation for this model

Consider a univariate discrete linear model : $z(k) = y(k) -(a* z(k-1) + b * z(k-2))$ where $y(k) = x(k) + \eta(k)$ $x(k) = s(k) + p*s(k-1) + q*s(k-2)$ is a Moving Average model of order 2. ...
2
votes
1answer
36 views

Gaussian Process Regression for piecewise linear response functions

I am performing Gaussian Process Regression (without noise) for response functions which are piecewise linear. My question: Does there exist a covariance function, such that sample paths from a ...
0
votes
0answers
21 views

Too small baseline predictors

I'm trying to implement a recommender system, based on SVD-algorithm. I have a matrix with binary rates, i.e. 0 and 1. This matrix is very sparse. I'm using a formula for learning process: ...
0
votes
1answer
60 views

Hessian for linear regression with regularization

I'm using matlab to solve a regularized linear regression via the fminunc() function. The cost function is from the standford machine learning class. It's pretty slow so and I think it could be sped ...
1
vote
0answers
37 views

Calculating Mutual Information for feature selection

In order to determine the importance of some individual features coming from labelled time series, I am trying to calculate the Mutual Information (as showed in "Who do you sync you are?: smartphone ...
0
votes
0answers
10 views

Automated Structured Data Extraction

I am following a paper on Structured Data Extraction titled : Finding and Extracting Data Records from Web Pages, M. Alvarez, A. Pan et al. I am writing code for same in ruby. The data region ...
0
votes
0answers
26 views

Gaussian process log marginal likelihood

I am trying to use a Gaussian process to predict some outcomes. I'm using the squared exponential co-variance function and 0 mean. My inputs are 4 dimensional vectors and I am using Maple for all ...
0
votes
1answer
45 views

Should cross-validation be used to provide the final parameters, or just to compare models?

In Andrew Ng's Coursera class on Machine Learning, we learned to use a Gaussian distribution $p(x)=\prod^n_{j=1}p(x_j,μ_j,σ^2_j)$ to detect anomalous examples when $p(x)<\epsilon$ where $x_j$ are ...
0
votes
0answers
17 views

Adding hard conditions/rules during machine learning

I am training a LinearSVM for a text classification task. In my data I have some instances that can be classified as 1 but I want to set some hard conditions or rules during the learning stage and ...
0
votes
1answer
33 views

semi-supervised metric learning

I was reading an article about metric learning lately. http://arxiv.org/pdf/1306.6709.pdf. In the paper, the author indicates that there are three types of metric learning paradigms, i.e., the full ...
2
votes
1answer
44 views

Relationship between VC dimension and degrees of freedom

I'm studying machine learning and I feel there is a strong relationship between the concept of VC dimension and the more classical (statistical) concept of degrees of freedom. Can anyone explain such ...
0
votes
1answer
21 views

How to preprocess a large sparse matrix and unbalanced classes in machine learning

I have a large very sparse matrix with 1000 columns and 15000 rows. It mainly contains zeros, the rest is integer values from 1-8. I'm limited to scikit-learn and ...
1
vote
1answer
46 views

Implementing Convolutional Neural Network - Problems

Recently I have started to implement my own Convolutional Neural Network. I have few questions. I will talk with reference to an example, so that we all remain on the same page. Suppose, input: ...
0
votes
0answers
30 views

Resources for machine learning for time-dependent data

For the past year, I have spent the majority of my free time learning a variety of ML techniques (boosting, random forests, neural nets, SVMs etc.), but I have not been able to find a lot of material ...
0
votes
0answers
23 views

Does the total squared error necessarily not increase after each iteration of stochastic gradient descent?

I know that for batch gradient descent $$\sum_{\text{x in data set}} (h(x) - t(x))^2$$ will not increase after each iteration, where $h(x)$ is the model's output on $x$ and $t(x)$ is the target ...
0
votes
0answers
17 views

High AUC but low sen/spe, am I doing something wrong?

My results for Upsampling and SMOTE gives extremely high AUCs. Is that normal given my specificity and sensitivity are low? I optimized the threshold to provide max sensitivity and specificity. So I ...
1
vote
1answer
22 views

Penalizing common words in LDA analysis

I have a corpus I want to perform an LDA on, however it has very few total words and some words occur extremely often. I want to penalize these words. A tfidf at first seemed intuitive (and I have ...
0
votes
1answer
23 views

How can a label have a precision of 100% and specifity of 100% as well?

I am using the Java library Mulan to do Multi-Label classification. My feature vector consists of 144 labels named "Slot1" to "Slot144". Each label can have the values 1 or 0. I know for a fact that ...
0
votes
1answer
36 views

Error: too many ties in knn in R

I am trying to use the KNN algorithm from the class package in R. I have used it before on the same dataset, without normalizing one of the features, but it ...
0
votes
0answers
21 views

How to learn from dataset where there are repeated instances but with different values?

I'm facing an issue with my dataset. The issue is that I have for example 1000 instances (customers or students) but I have 4000 lines/records because some of the instances are presented in several ...
0
votes
1answer
46 views

Random Forest: Different performance between training set and test set?

I'm a newbie learning Random Forest. When I use this method to predict my outcome and check with the same data set (training set), I see that the model fits almost perfectly the data. But when I ...
0
votes
1answer
29 views

How to interpret the relation or interaction between two variables or features in general for classification?

Suppose I have several features like $X_1, X_2, X_3$ etc. In my model, I have to know whether $X_1$ and $X_2$ will have an impact together. I read somewhere we can make a new feature by $( X_1*X_2 )$, ...
1
vote
0answers
31 views

Data distribution problem

I am having the transaction dataset which contains 95% or user initiated transactions and 5% fraud transactions. When I am fitting the logistic regression model, it gives me bias prediction - as most ...
0
votes
1answer
32 views

How to use Particle Swarm Optimization for finding hyper-parameters of Support Vector Regression?

I want to use Particle Swarm Optimization (PSO)for finding hyper parameters of a support vector regression problem. Initially I tried to find the same using grid search method,but the Matlab code is ...
1
vote
1answer
34 views

Stepwise regression for Bayesian models

Why isn't stepwise regression, like backward elimination, used for Bayesian models? What is generally used to find insignificant variables in bayesian methods? Or does one simply not worry about ...
0
votes
0answers
27 views

What are outlier measures for regression problems?

I want to detect outliers automatically and some how eliminate effect of them in a regression problem. In fact I don't even want to detect outliers. I need to just eliminate or minimize the effect of ...
2
votes
1answer
36 views

Chi Squared Kernel and Faster implementation

There is a good implementation of Chi-Squared Kernel in http://www.vlfeat.org/matlab/vl_alldist2.html But this implementation is very slow when input data is huge. This implementation doesn't accept ...
1
vote
1answer
28 views

Comparing and evaluating various machine learning methods

I am not expert in this area so please bare with me. Is it possible to somehow evaluate the success rate of machine learning algorithm/methods. I suppose it could be done this way: Give a various ML ...
0
votes
0answers
19 views

Loss function for Random forest

I am working on a random forest model in R and want to use a different loss function from the default. Does random forest implementation in R allow for arbitrary loss functions?
0
votes
0answers
52 views

Significance of explanatory variables in Bayesian models

I was wondering if there is a general way to handle parameters of which posterior distributions include zero. Should one remove these parameters and refit the model? E.g. You fit a regression model ...
1
vote
0answers
32 views

chi square test for large data sets

I use the Chi-square test for feature selection. I use it only when all entries in the contingency table are greater then 5. Is that the correct approach statistically? What happens for example, if ...
1
vote
0answers
30 views

How to run an iteration with optim in R? [closed]

Can anyone tell me how to run an iteration in R? I have been trying an iteration but without success. The reduced form of the code is given below. I will explain briefly what i am trying to acheive ...
0
votes
0answers
30 views

How to implement Kernel density estimation in multivariate/3D

I have dataset like the following fromat and im trying to find out the Kernel density estimation with optimal bandwidth. ...