Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
1answer
22 views

Neural Netwok and coadaptation problem

Dropout proposed by Hinton is said to prevent co-adaptation. My question is on co-adaptation. How can I measure the coadaptation that occurs in a MLP that does not use a drop out?
0
votes
0answers
13 views

maxout and Rectified linear unit in Neural networks [on hold]

In the context of neural networks: What is the difference between maxout and rectified linear units?
1
vote
1answer
37 views

Is there any neural network whose output can be probabilistic, just like multi-class logistic regression?

I want to add nonlinear character into multi-class logistic regression. I know kernel logistic regression can do it. Is there any kind of neural network which has similar characteristic?
0
votes
0answers
27 views

Threshold on tanh or sigmoid in Convolutional neural network

I have read several papers on Convolutional Neural Nets but I am yet to come across any that has used thresholds on tanh or sigmoid to decide whether the neuron will fire or not. Obviously this works ...
1
vote
1answer
34 views

Problem with getting neural network learned to calculate XOR

I am learning about neural networks. I found a course on Coursera about machine learning https://www.coursera.org/course/ml . What I am trying to implement is a neural network to calculate logical ...
0
votes
1answer
43 views

What gradient descent method is better for convolutional neural network?

Let's say we want to train a convolutional neural network, what gradient descent method works better? (1) Batch gradient descend (2) stochastic gradient descent
0
votes
1answer
43 views

Can a neural network with random connections still work correctly?

Let's say we have a neural network with n layers where connections do not simply go from layer i to layer i+1, but can go from any layer i to any layer k such that k > i. For example; connections from ...
0
votes
1answer
41 views

pruning Neural Network

Since a feedforward NN with a logistic function as activation function is not linear, does it make sense to reduce variables first with principal components or discriminant analysis? Because ...
3
votes
3answers
79 views

Explanation on a Minsky's critique on statistical learning related to XOR

I was listening to the first session of society of Minds by Minsky (2011) and he mentions at some point around minute 48 the following: "...lots of statistical learning tools is good for lots of ...
0
votes
1answer
16 views

Viability of software dev - Use of and requirements of NN

Hello I would like to know this two things regarding the viability of producting a software, so: 1) Are available on internet some OCR libraries for free? Can I train my own NN having only a laptop? ...
0
votes
0answers
14 views

How to classify a unbalanced dataset by Convolutional Neural Networks (CNN)?

I have a unbalanced dataset in a binary classification task, where the positives amount vs negatives amount is 0.3% vs 99.7%. The gap between positives and negatives are huge. When I train a CNN with ...
-1
votes
2answers
39 views

What is the difference between a neural network and a perceptron?

Is there any difference between the terms "neural network" and "perceptron"?
0
votes
1answer
25 views

Is there AUC for neural network?

I am confused about how to calculate AUC for neural network with a softmax classifier. For example, I know that for SVM, we can change the threshold value and determine the AUC. WHat about in neural ...
1
vote
1answer
18 views

Is there any difference between training a stacked autoencoder and a 2-layers neural network?

Let's say I am writing an algorithm for building 2-layers stacked autoencoder and 2-layers neural network. Are they the same things or difference? What I understand is that when I build a stacked ...
0
votes
0answers
31 views

Is the gradient computation in the word2vec implementation actually wrong?

In the paper "Efficient Estimation of Word Representations in Vector Space", it is stated that "All models are trained using stochastic gradient descent and backpropagation": ...
0
votes
0answers
5 views

Error in model.frame.default … invalid type (list) for variable [migrated]

I'm fairly new to R and I'm trying to create a model to work on Kaggle's Facial Keypoint Detection sample project. The ultimate issue is that creating any model (I'm trying a neural net using the ...
1
vote
2answers
49 views

Should image classifier be trained using colormap pixels or the actual value?

For example, I have a population density map of a 100 x 100 km square region. Each part of the rectangular region represents the population density i.e. (1,1) -> 128 people, (100,100) -> 50 people ...
3
votes
1answer
48 views

R Deep Learning with H2O vs neuralnet package

I had a fairly open ended question. I've been looking at deep learning architectures for neural networks for classification in R. A few packages came up, neuralnet, ...
3
votes
1answer
19 views

How to determine the number of convolutional operators in CNN?

In computer vision task, such as object classification, with Convolutional Neural Networks (CNN), the network provides an appealing performance. But I'm not sure how to set up the parameters in ...
2
votes
0answers
53 views

How does Krizhevsky's '12 CNN get 253,440 neurons in the first layer?

In Alex Krizhevsky, et al. Imagenet classification with deep convolutional neural networks they enumerate the number of neurons in each layer (see diagram below). The network’s input is ...
3
votes
1answer
46 views

Should I ever manually modify training data?

TL;DR: I'm reviewing a computer vision + machine learning module that someone else wrote, and I've discovered that she is manually cleaning up training data. Is that ever a good idea? The Details ...
0
votes
1answer
27 views

Gibbs Sampling for Boltzmann Machines

David Mac Kay, in his book on machine learning talks about Boltzmann machines, and on pg. 3 here http://www.inference.phy.cam.ac.uk/itprnn/ps/521.526.pdf He says "the second equation ...
3
votes
1answer
41 views

maximum a posteriori vs squared loss

I am unclear about max a posteriori and squared loss. Let me assume I have $N$ images and $\mathbf{y}_i$ is the label of the image $i$, where, $\mathbf{y}_i\in \mathbb{R}^{C\times 1}$ - a binary ...
0
votes
1answer
23 views

training and testing an artificial neural network method

Is it alright to apply different epoch numbers to train/test a ANN method ( i.e. set number of iterations/epoch for training mode, and then set another different number of epoch for testing mode) ?
0
votes
1answer
26 views

Neural network over-fitting

I've learned that over-fitting can be detected by plotting the training error and the testing error versus the epochs. Like in: I've been reading this blogpost where they say the neural network, ...
0
votes
1answer
37 views

Is it useful to normalize or standardize if your features are binary?

I have binary features that I am training on a machine learning model. For example: <0, 0, 1> <1, 0, 0> I am going to train a neural network on this ...
2
votes
3answers
50 views

Find Neural Network Inputs Given Outputs

I've trained a neural network with two inputs, a single hidden layer with two neurons, and one output using a bipolar sigmoid activation function. If a single input is known, how would I determine the ...
0
votes
0answers
2 views

Predict method of caret package gives error :Error in models[[1]]$trainingData$.outcome [migrated]

I am new to neural network and caret package and struggling with an issue. I am training the model using train() method of caret package with method='nnet', and getting model fit without any error. ...
0
votes
1answer
32 views

How to prove absolute lack of correlation

I have a huge dataset of 17 variables. I intended to use 15 of those to predict the 17th, and I could not find any model (ANN) to do so. I know that one of those variables definitely predicts the ...
4
votes
2answers
137 views

Autoencoders can't learn meaningful features

I have 50,000 images such as these two: They depict graphs of data. I wanted to extract features from these images so I used autoencoder code provided by Theano (deeplearning.net). The problem ...
0
votes
1answer
12 views

Compare convergence of optimization methods

I need to quantify how 2 optimization methods differ in convergence. When training a neural network I get the following plots, which show an error function after each gradient update. I think the ...
0
votes
0answers
6 views

Theano MLP is not predicting properly [migrated]

Hi guys belowe my MLP theano code from sklearn.metrics import classification_report import os,sys,time,numpy,theano,cPickle,gzip import theano.tensor as T import numpy as np filefolder = ...
0
votes
1answer
25 views

Using both L2 regularization and early stopping when training ANN's

I came across an ANN that's used for approximating a noisy sine function. On one hand it uses a validation set for early stopping, and then again uses the same validation set for fine tuning the ...
2
votes
1answer
39 views

Multi-class neural network error derivative with respect to weights calculation

Suppose I have $N$ training examples and there are $K$ classes and the targets have a $1$ of $K$ encoding Let $t_k^n$ denote the kth component of the nth training target Let $x^n$ denote the the ...
0
votes
0answers
16 views

Is Multi-output of SVM Feasible for my research?

I am working on a decision making system, something about concert prices prediction to maximize the profit. Because it is multi-output, now data mining algorithm I know only neural network is suitable ...
2
votes
2answers
29 views

backpropagation - bias nodes and error

I am implementing the stochastic gradient descent version of backpropagation from Tom Mitchell's Machine Learning book which has the steps for each training instance $\langle\vec{x},\vec{t}\rangle$: ...
0
votes
1answer
54 views

What is maxout in neural network?

Anyone can explain what does maxout layer do in neural network? How to perform it? What does it different to normal activation function? I try to read the 2013 "Maxout Network" paper by Goodfellow et ...
0
votes
1answer
43 views

How to standardize text data for training Neural Networks?

I want to train neural network with text data(natural language) as input for classification purpose. One way for standardizing text data for neural network is to use N-GRAM/SKIP-GRAM representation ...
0
votes
1answer
34 views

XOR backpropagation convergence

I've implemented 3 supervised training algorithms: rprop, online- and batch backprop with momentum. I have the simple XOR test, and I measured how many times they converge out of N iterations. My ...
5
votes
1answer
175 views

Deep learning algorithm

What's the difference between deep belief network and deep convex network?
2
votes
1answer
35 views

How can recurrent neural networks be used for sequence classification?

RNN can be used for prediction, or sequence to sequence mapping. But how can RNN be used for classification? I mean, we give a whole sequence one label.
0
votes
1answer
69 views

Types of artificial intelligence with good results [closed]

I have been looking into artificial intelligence for some time now. I am wondering what branches are still in active research and have some good/interesting results. The two that I have looked in so ...
0
votes
1answer
26 views

ImageNet: what does top-five error means?

One of the evaluation method for ImageNet Competition (classify 1,000 categories images) is top-5 error, what does that mean? See: http://www.image-net.org/challenges/LSVRC/
0
votes
0answers
24 views

Manually Calculating Tan Sigmoid & Softmax

I'm putting together an app for my dissertation which will take a set of 34 inputs and give back a diagnosis. I've built the ANN in Matlab and trained and tested it. I am then taking the weights and ...
0
votes
0answers
22 views

Building a single-layer neural network for number recognition

I'm trying to create a very simple neural net in Javascript for a school project. The goal is to have the net identify a number drawn by the user on a square grid. Currently the size of this grid is ...
0
votes
0answers
39 views

One-class SVM vs NN with backprop… Or is there something better?

I'm pretty new to unary classification, so I've been playing around with different approaches to one-class document classification in Python. NN seemed promising at first, but has some undesirable ...
0
votes
0answers
13 views

The neural networks that are the best at invariance

There is a great number of types of neural networks. Some are better at handling invariance, some are worse, and some are not capable of it at all. I don't know any except the ASSOM, and the broad ...
0
votes
0answers
16 views

Can you take a DNN that was trained without regularization, and continue training it with regularization?

If I've trained a DNN with out any regularization methods (e.g. weight decay, dropout etc.) and reached a good training error, can I somehow take that learned net and fine tune it with regularization? ...
1
vote
0answers
32 views

number of feature maps in convolutional neural networks

When learning convolutional neural network, I have questions regarding the following figure. 1) C1 in the layer 1 has 6 feature maps, does that mean there are six convolutional kernals? Each ...
0
votes
0answers
29 views

Convolutional neural network for time series?

I would like to know if there exist a code to train a convolutional neural net to do time-series classification. I have seen some recent papers ...