Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
0answers
11 views

classify with 3 class [on hold]

How do I calculate average rate error using Bayes and neural network classification, for example, on the three classes in Fisher's iris data?
0
votes
1answer
11 views

What does training correlation coefficient means in ANN prediction

I'm a beginner in statistics with limited knowledge. I'm reading up on neural networks to predict outputs using inputs. I understand that a neural network has to be trained to produce the Least mean ...
0
votes
0answers
14 views

Graphically, how does the non-linear activation function project the input onto the classification space?

I am finding a very hard time to visualize how the activation function actually manages to classify non-linearly separable training data sets. Why does the activation function (e.g tanh function) ...
0
votes
0answers
6 views

What guidelines should be followed for using Neural Networks with sparse inputs

I have extremely sparse inputs, e.g. locations of certain features in an input image. Further each feature can have multiple detections (not sure if this will have a bearing on the design of the ...
1
vote
1answer
23 views

How to apply the output layer function in a neural network

I am implementing a Neural Network in a somewhat different fashion. I train my neural network locally using a small subset, and export the weights. My goal is to test the neural network in a ...
0
votes
1answer
11 views

Different activation function in nnet R

Can different activation functions be specified for hidden and output layers for any of the R neural network packages?
1
vote
2answers
24 views

What does “permutation invariant” mean?

I have seen a term "permutation invariant" version of the MNIST digit recognition task. What does it mean?
1
vote
2answers
50 views

What're the differences between PCA and autoencoder?

Both PCA and autoencoder can do demension reduction, so what are the difference between them? In what situation I should use one over another?
0
votes
2answers
21 views

Can a neuron output a value other than 1 or 0

I need to know whether a neuron output can take any value other than a binary value. i.e 0 or 1 1 or -1 except values like these can a neuron output any ...
0
votes
0answers
16 views

How to improve ANN results?

I have a 10 by 57300 matrix as an input, and a 1 by 57300 matrix as an output that only includes 0 and 1.I tried to train neural network with feed-forward back propagation and layer recurrent back ...
2
votes
1answer
21 views

What does pre-training mean in deep autoencoder?

I am confused by the term "pre-training". What does it mean in deep autoencoder? And how does it help improving the performance of autoencoder? (I know this term comes from Hinton 2006's paper: ...
0
votes
0answers
18 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron ...
0
votes
0answers
8 views

Why did convolutional nets perform so much better than normal MLPs for MNIST?

MNIST (http://yann.lecun.com/exdb/mnist/) is a database of handwritten digits. Every digit is already segmented, centered and of the same scale. On the link above, one can see how good different ...
0
votes
0answers
18 views

Do we need to do sigmoid after pooling in convolutional neural network?

I understand we need to do sigmoid transformation after convolution step in building convolutional neural network(CNN). Do we need that also after pooling step? Like: Let assume: CL = convolution ...
0
votes
0answers
25 views

What is the correct architecture for convolutional neural network?

I have seen several different architectures for convolutional neural network (CNN). I am confused which one is the standard and how do I decide what to use. I am not confused by the number of layers ...
2
votes
1answer
22 views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
0
votes
1answer
27 views

In Convolutional neural network, what does fully-connected layer mean?

There are convolution layers, pooling layers, and possibly a classifier layer (e.g. softmax layer) in convolutional neural network(CNN). I heard that there is also a fully-connected layer, what is ...
0
votes
1answer
22 views

Deep neural network fine-tuning with Random Forest being the last layer

After layerwise pretraining (typically unsupervised) the dnn, we tend to fine tune it in a supurvised manner. I know how to do the fine-tuning if the last layer is a softmax classifer, but how to do ...
1
vote
0answers
28 views

Can a perceptron be modified so as to converge with non-linearly separable data?

Normally, a perceptron will converge provided data are linearly separable. Now if we select a small number of examples at random and flip their labels to make the dataset non-separable. How can we ...
1
vote
0answers
16 views

recursive neural network with binary output- new question

I have a question about binary classification with recursive neural networks (I already read other posts on this topic): Let's consider Elman's approach that generates output $y=g(v\times s)$ where ...
0
votes
0answers
6 views

How big of an effect can I expect when starting to modify the structure of neural net to fit my data better?

I downloaded theano and started playing with it, trying to write a software to identify images with cars. Now I've played around with it a bit and got an decent'ish error rate or about 30%, but it's ...
6
votes
2answers
71 views

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? ...
1
vote
0answers
30 views

How can I make sure that an LDA implementation works?

I am currently experimenting with neural nets for classification of on-line handwritten data (hence: not pixels, but time series data). To do so, I use several toolkits (internal development of my ...
0
votes
0answers
9 views

How we can statistically compare performance of two models before and after outlier detection?

As you know we can use Mcnemar's test to compare performance of two models in binary classification problem. But in my case i ...
0
votes
0answers
20 views

What is the use of convolution and pooling in convolutional neural network

I am confused the use of convolution and pooling in convolutional neural network(CNN). I know pooling is basically summarizing the adjacent features, which can greatly reduced the features size of an ...
1
vote
0answers
35 views

Performance of algorithms using Jacobian matrix on large data sets

Some ANN learning algorithms like Gauss-Newton, Levenberg-Marquardt require creation of a Jacobian matrix. Having studied this I found out that the Jacobian matrix is huge (unless I misunderstood ...
1
vote
1answer
15 views

Weight Decay in Neural Neural Networks Weight Update and Convergence

I have a neural network (That I created using java) for a class assignment that is working when I do not use any weight decay value, but when I use a value greater than or equal to .001, my accuracy ...
0
votes
0answers
33 views

Using third validation set in Cross Validation?

(Note there's 2 paragraphs of background information before I get to the question) I've got a Neural Network classifier, trained with an EA to classify data. I previously used a holdout framework ...
0
votes
0answers
179 views

In Graph Transformer Networks, which parameters are tuned during back-propagation?

Referring to the milestone paper "Gradient-Based Learning Applied to Document Recognition" of LeCun, Bottou, Bengio and Haffner, which parameters of the graph transformer networks for global training ...
0
votes
0answers
27 views

Neural Networks sigmoid activation with bias updates

I am trying to figure out if I am creating an artificial neural network using the sigmoid activation function and using bias correctly. I want one bias node to input to all hidden nodes with static ...
0
votes
0answers
13 views

NARX model to predict future values

I have this problem , where I have to predict a value of a indicator which depends on 270 other predictor variables. I read the time series modelling and prediction on MATLAB , which took the example ...
0
votes
1answer
24 views

Very different Neural Network test errors for same architecture

So I'm doing a time series prediction, and assessing the capability of the ANN to predict that time series. I am using Matlab's neural network toolbox functions, and the training parameters are the ...
0
votes
0answers
40 views

What is a convolutional neural network

I have been studying neural networks and I recently found out about deep learning and convolutional neural networks. Can someone give me a newbie introduction to convolutional neural networks, what ...
0
votes
1answer
26 views

How well do Convolutional neural networks in other image domains?

I was recently trying out caffe and learning about CNN. So far I have seen that the model used by Krizhevsky performs really well in natural images. However I wanted to know how these models or CNN ...
0
votes
0answers
20 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
1
vote
0answers
30 views

What kind of weight values should a restricted Boltzmann machine have?

I designed a Gaussian (Gaussian distributed visible layer) - Bernoulli (binary distributed) RBM model (for reference, see: Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines, pdf) ...
0
votes
0answers
12 views

Neural Network & Image recognition - preprocessing

I have a project where I´m supposed to recognize ordinary data written numbers (1-10) from images as well as some geometrical shapes (rectangular, triangle, circle, etc.). However, I´m not sure about ...
4
votes
2answers
69 views

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here ...
1
vote
0answers
45 views

disadvantages of Neural network method

Hello Dear Researchers! I want to list the advantages and disadvantages of Neural network methods for classification or estimation purposes. I have already found the advantages of NN method in many ...
0
votes
1answer
34 views

Neural Net Hidden Layer

I have done some searching on the topic and I know the answer is mainly "it depends". BUT, I haven't found anything stating general relationships regarding size of data and number of hidden layers. ...
0
votes
1answer
15 views

Tanh activation function and sparsity constraint

According to Lecun's paper "effient backprop" [1] the tanh activation function should be preferred over the logistic activation function for the hidden units in neural networks. For the tanh units ...
0
votes
0answers
12 views

What should be the fitness function while using Particle Swarm optimisation

I am using Particle Swarm Optimisation for optimising the parameters of a Neural network (for multi-class classification problem). But what should be the fitness function for it ? I have tried ...
0
votes
1answer
20 views

Requirements for a valid neural network activation function?

What rules define a valid neural network activation function, excluding biological plausibility? What set of principles do softmax, rectified linear units, hyperbolic tangent, sigmoid, etc. follow? ...
0
votes
0answers
23 views

repeating rare examples in unbalanced data classification

So I'm trying to train a neural network for a rare event detection. based on that i have like 1000 times more examples for non-target (everything else) examples that i have for target examples. So i ...
2
votes
1answer
29 views

Comprehensive list of activation functions in neural networks with pros/cons

Are there any reference document(s) that give a comprehensive list of activation functions in neural networks along with their pros/cons (and ideally some pointers to publications where they were ...
1
vote
0answers
16 views

Why can't I approximate sinusoid with tanh activation function?

I've been playing with this applet for learning about backpropagation: http://neuron.eng.wayne.edu/bpFunctionApprox/bpFunctionApprox.html What continues to confuse me, is why I can't approximate ...
0
votes
1answer
55 views

Logistic Regression, SVM or NN?

Just attended Andrew Ng’s online course on ML and although I’ve understood the methods I seem to be missing the intuition on where to apply them in terms of classification problems. What are the ...
0
votes
0answers
10 views

Neural Network hidden layer proportionality

I am working on Proportionality on Multiple Hidden layer . My application area is Handwritten Pin code Recognition. I had used five hidden layer for better accuracy. I know time complexity will ...
3
votes
2answers
90 views

Newbie to neural networks

Just starting to play around with Neural Networks for fun after playing with some basic linear regression. I am an English teacher so don't have a math background and trying to read a book on this ...
1
vote
4answers
175 views

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between convolutional neural networks (CNN), restricted Boltzmann ...