Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
1answer
28 views

Multi Output Neural Networks

Up until know I only used neural networks to classify a single output, I set one output neuron for each class and check which neuron has the highest/lowest activation. What I am trying to do is to ...
0
votes
0answers
15 views

Early stopping methods for ANN applied to series prediction

Could anyone give advice or links to advice on early stopping methods for ANN trained with back prop applied to time series prediction? I know some methods for classification tasks but don't the ...
0
votes
0answers
23 views

Neural Network for hand written digit recognition

I have create the neural network with three layers. 1 layer - 500 inputs 2 layer - 500 inputs 3 layer - 10 output classes. I have synthesized the ...
0
votes
1answer
18 views

Convolutional neural network with non-image input data

Can CNNs be used with input data which is not an image? The reason I'm asking is because the original image is often clipped in size because of border effects when doing the convolution. But if the ...
1
vote
0answers
16 views

How do we get/define filters in convolutional neural networks?

How do i obtain filters from convulutional neural network(CNN)? My idea is something like this: Do random images of the input images (28x28) and get random patches (8x8). Then use autoencoders to ...
1
vote
1answer
33 views

Does Deep network (e.g. # of hidden layer=2) always better than shallow network (i.e. # of hidden layer=1)?

I attempted to build a deep network (e.g. deep autoencoder) for some object classification, my result showed that the deep networks is worst than shallow network. However, from what I have read from ...
0
votes
0answers
21 views

classify with 3 class [closed]

How do I calculate average rate error using Bayes and neural network classification, for example, on the three classes in Fisher's iris data?
0
votes
1answer
12 views

What does training correlation coefficient means in ANN prediction

I'm a beginner in statistics with limited knowledge. I'm reading up on neural networks to predict outputs using inputs. I understand that a neural network has to be trained to produce the Least mean ...
0
votes
0answers
16 views

Graphically, how does the non-linear activation function project the input onto the classification space?

I am finding a very hard time to visualize how the activation function actually manages to classify non-linearly separable training data sets. Why does the activation function (e.g tanh function) ...
0
votes
0answers
8 views

What guidelines should be followed for using Neural Networks with sparse inputs

I have extremely sparse inputs, e.g. locations of certain features in an input image. Further each feature can have multiple detections (not sure if this will have a bearing on the design of the ...
1
vote
1answer
25 views

How to apply the output layer function in a neural network

I am implementing a Neural Network in a somewhat different fashion. I train my neural network locally using a small subset, and export the weights. My goal is to test the neural network in a ...
0
votes
1answer
11 views

Different activation function in nnet R

Can different activation functions be specified for hidden and output layers for any of the R neural network packages?
1
vote
2answers
30 views

What does “permutation invariant” mean?

I have seen a term "permutation invariant" version of the MNIST digit recognition task. What does it mean?
1
vote
2answers
54 views

What're the differences between PCA and autoencoder?

Both PCA and autoencoder can do demension reduction, so what are the difference between them? In what situation I should use one over another?
0
votes
2answers
21 views

Can a neuron output a value other than 1 or 0

I need to know whether a neuron output can take any value other than a binary value. i.e 0 or 1 1 or -1 except values like these can a neuron output any ...
0
votes
0answers
16 views

How to improve ANN results?

I have a 10 by 57300 matrix as an input, and a 1 by 57300 matrix as an output that only includes 0 and 1.I tried to train neural network with feed-forward back propagation and layer recurrent back ...
2
votes
1answer
23 views

What does pre-training mean in deep autoencoder?

I am confused by the term "pre-training". What does it mean in deep autoencoder? And how does it help improving the performance of autoencoder? (I know this term comes from Hinton 2006's paper: ...
0
votes
0answers
19 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron ...
0
votes
0answers
10 views

Why did convolutional nets perform so much better than normal MLPs for MNIST?

MNIST (http://yann.lecun.com/exdb/mnist/) is a database of handwritten digits. Every digit is already segmented, centered and of the same scale. On the link above, one can see how good different ...
0
votes
0answers
20 views

Do we need to do sigmoid after pooling in convolutional neural network?

I understand we need to do sigmoid transformation after convolution step in building convolutional neural network(CNN). Do we need that also after pooling step? Like: Let assume: CL = convolution ...
0
votes
0answers
25 views

What is the correct architecture for convolutional neural network?

I have seen several different architectures for convolutional neural network (CNN). I am confused which one is the standard and how do I decide what to use. I am not confused by the number of layers ...
2
votes
1answer
26 views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
0
votes
1answer
30 views

In Convolutional neural network, what does fully-connected layer mean?

There are convolution layers, pooling layers, and possibly a classifier layer (e.g. softmax layer) in convolutional neural network(CNN). I heard that there is also a fully-connected layer, what is ...
0
votes
1answer
23 views

Deep neural network fine-tuning with Random Forest being the last layer

After layerwise pretraining (typically unsupervised) the dnn, we tend to fine tune it in a supurvised manner. I know how to do the fine-tuning if the last layer is a softmax classifer, but how to do ...
1
vote
0answers
29 views

Can a perceptron be modified so as to converge with non-linearly separable data?

Normally, a perceptron will converge provided data are linearly separable. Now if we select a small number of examples at random and flip their labels to make the dataset non-separable. How can we ...
1
vote
0answers
17 views

recursive neural network with binary output- new question

I have a question about binary classification with recursive neural networks (I already read other posts on this topic): Let's consider Elman's approach that generates output $y=g(v\times s)$ where ...
0
votes
0answers
7 views

How big of an effect can I expect when starting to modify the structure of neural net to fit my data better?

I downloaded theano and started playing with it, trying to write a software to identify images with cars. Now I've played around with it a bit and got an decent'ish error rate or about 30%, but it's ...
6
votes
2answers
82 views

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? ...
1
vote
0answers
31 views

How can I make sure that an LDA implementation works?

I am currently experimenting with neural nets for classification of on-line handwritten data (hence: not pixels, but time series data). To do so, I use several toolkits (internal development of my ...
0
votes
0answers
13 views

How we can statistically compare performance of two models before and after outlier detection?

As you know we can use Mcnemar's test to compare performance of two models in binary classification problem. But in my case i ...
0
votes
0answers
24 views

What is the use of convolution and pooling in convolutional neural network

I am confused the use of convolution and pooling in convolutional neural network(CNN). I know pooling is basically summarizing the adjacent features, which can greatly reduced the features size of an ...
1
vote
0answers
37 views

Performance of algorithms using Jacobian matrix on large data sets

Some ANN learning algorithms like Gauss-Newton, Levenberg-Marquardt require creation of a Jacobian matrix. Having studied this I found out that the Jacobian matrix is huge (unless I misunderstood ...
1
vote
1answer
17 views

Weight Decay in Neural Neural Networks Weight Update and Convergence

I have a neural network (That I created using java) for a class assignment that is working when I do not use any weight decay value, but when I use a value greater than or equal to .001, my accuracy ...
0
votes
0answers
35 views

Using third validation set in Cross Validation?

(Note there's 2 paragraphs of background information before I get to the question) I've got a Neural Network classifier, trained with an EA to classify data. I previously used a holdout framework ...
0
votes
0answers
192 views

In Graph Transformer Networks, which parameters are tuned during back-propagation?

Referring to the milestone paper "Gradient-Based Learning Applied to Document Recognition" of LeCun, Bottou, Bengio and Haffner, which parameters of the graph transformer networks for global training ...
0
votes
0answers
27 views

Neural Networks sigmoid activation with bias updates

I am trying to figure out if I am creating an artificial neural network using the sigmoid activation function and using bias correctly. I want one bias node to input to all hidden nodes with static ...
0
votes
0answers
16 views

NARX model to predict future values

I have this problem , where I have to predict a value of a indicator which depends on 270 other predictor variables. I read the time series modelling and prediction on MATLAB , which took the example ...
0
votes
1answer
24 views

Very different Neural Network test errors for same architecture

So I'm doing a time series prediction, and assessing the capability of the ANN to predict that time series. I am using Matlab's neural network toolbox functions, and the training parameters are the ...
0
votes
0answers
41 views

What is a convolutional neural network

I have been studying neural networks and I recently found out about deep learning and convolutional neural networks. Can someone give me a newbie introduction to convolutional neural networks, what ...
0
votes
1answer
26 views

How well do Convolutional neural networks in other image domains?

I was recently trying out caffe and learning about CNN. So far I have seen that the model used by Krizhevsky performs really well in natural images. However I wanted to know how these models or CNN ...
0
votes
0answers
21 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
1
vote
0answers
30 views

What kind of weight values should a restricted Boltzmann machine have?

I designed a Gaussian (Gaussian distributed visible layer) - Bernoulli (binary distributed) RBM model (for reference, see: Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines, pdf) ...
0
votes
0answers
12 views

Neural Network & Image recognition - preprocessing

I have a project where I´m supposed to recognize ordinary data written numbers (1-10) from images as well as some geometrical shapes (rectangular, triangle, circle, etc.). However, I´m not sure about ...
4
votes
2answers
77 views

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here ...
1
vote
0answers
46 views

disadvantages of Neural network method

Hello Dear Researchers! I want to list the advantages and disadvantages of Neural network methods for classification or estimation purposes. I have already found the advantages of NN method in many ...
0
votes
1answer
34 views

Neural Net Hidden Layer

I have done some searching on the topic and I know the answer is mainly "it depends". BUT, I haven't found anything stating general relationships regarding size of data and number of hidden layers. ...
0
votes
1answer
15 views

Tanh activation function and sparsity constraint

According to Lecun's paper "effient backprop" [1] the tanh activation function should be preferred over the logistic activation function for the hidden units in neural networks. For the tanh units ...
0
votes
0answers
12 views

What should be the fitness function while using Particle Swarm optimisation

I am using Particle Swarm Optimisation for optimising the parameters of a Neural network (for multi-class classification problem). But what should be the fitness function for it ? I have tried ...
0
votes
1answer
20 views

Requirements for a valid neural network activation function?

What rules define a valid neural network activation function, excluding biological plausibility? What set of principles do softmax, rectified linear units, hyperbolic tangent, sigmoid, etc. follow? ...
0
votes
0answers
25 views

repeating rare examples in unbalanced data classification

So I'm trying to train a neural network for a rare event detection. based on that i have like 1000 times more examples for non-target (everything else) examples that i have for target examples. So i ...