Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
2answers
24 views

No regularisation term for bias unit in neural network

According to this tutorial on deep learning, weight decay (regularization) is not usually applied to the bias terms b why? What is significance (intuition) behind it?
0
votes
0answers
9 views

Recurrent vs Recursive Neural Networks: Which applies better for NLP?

So, we have Recurrent Neural Networks and Recursive Neural Networks. Both are usually denoted by the same acronym: RNN. According to Wikipedia, Recurrent NN are in fact Recursive NN, but I don't ...
0
votes
1answer
21 views

Neural Networks and Numeric Prediction

I'm new to machine learning and am trying to write a simple neural network that uses back-propagation. Now, so far I've successfully implemented my neural network to learn a boolean function. So for ...
0
votes
1answer
22 views

What is batch size in neural network?

I'm using Python Keras package for neural network. This is the link. Is batch_size equals to number of test samples? From ...
0
votes
0answers
9 views

LSTM end of input sign required?

I am coding an LSTM module. In non-linear input like one-hot-encoded words, you use an extra sign (like <eol>) at the end of the series before zero-padding to ...
0
votes
2answers
29 views

Traffic volume/flow prediction method

I have traffic volume data (Surrey City, CA) like this I wish to use Artificial neural network (Deep Learning) or ARIMA to predict traffic flow/volume of the urban area with the use of previous ...
0
votes
1answer
28 views

Derivative of softmax and squared error

I'm trying to understand the derivatives w.r.t. the softmax arguments when used in conjunction with a squared loss (for example as the last layer of a neural network). I am using the following ...
0
votes
0answers
18 views

What are the differences between autoencoder and t-SNE?

As far as I know, both autoencoder and t-SNE are used for nonlinear dimension reduction. What are the differences between them and why should I use one versus another? thanks!
0
votes
0answers
21 views

Best supervised neural network package for python

What is the best supervised neural network package for python? I found that sci-kit package only have unsupervised neural network.
0
votes
0answers
16 views

Tune a neural network and prevent overfitting

I'm using a neural network for the first time and I would like to know if I'm doing this right. I'm working with time series for 5 years, and in each year I have a total of 18 time series plus the ...
2
votes
0answers
25 views

Class probabilities in Neural networks

I use the caret package with multi-layer perception. My dataset consists of a labelled output value, which can be either A,B or C. The input vector consists of 4 ...
1
vote
2answers
29 views

Does Adding more neural units reduce the probability of trapping in a local minima?

Consider a multi-layer neural network that learn its weights with backpropagation (and gradient descent). Hence, there is a probability that we trap into a local minima. Will adding more neural units ...
1
vote
1answer
25 views

Accepted notation for sparsity/density of a network

Is there an accepted notation in the study of network structure for the measure of network sparsity and/or density? For example, I see $\rho$ is typically used for spectral radius, $\alpha$ for leak ...
1
vote
1answer
43 views

What should the divisor be in MSE?

The mean squared error for two vectors is calculated using $$mse(x, y) = \frac{1}{n}\sum_{i=1}^n(x_i-y_i)^2\text{, with }x, y \in \mathbb{R^n}\text{.}$$ That is straightforward, but I have a hard ...
0
votes
1answer
39 views

When would a 3-layer Neural Network perform better than a 2-layer with the same number of parameters?

I'm interested in knowing what is the benefit of having 3 fully-connected layers in a Neural Network instead of 2. Many deep Neural Networks such as ImageNet do this. Why is this superior as compared ...
0
votes
1answer
26 views

Target and output in neural networks

In ANN the output squeezed using sigmoid function so the result is always between 1 and -1. How am I supposed to calculate the error when the target value might be a big number? For example I'm ...
0
votes
1answer
57 views

Using Neural Networks to predict stock values

How are neural networks usually used to predict market evolution? My data consists of a set of pairs (time, value), taken at an interval of 15 minutes. My ideas so far are: I.Take 40 values (or ...
1
vote
1answer
35 views

Can an XOR gate be implemented using an ANN of linear transfer fucntions?

It is the case that XOR functions can be implemented using an ANN of logistic sigmoid functions. Is this also the case using linear transfer functions? Say if the output of the overall network is ...
0
votes
1answer
36 views

What's the best way to convert pair-wise ranking to a ranked list?

I have a set of ordered items A > B > C ... > F. For each element of the set I have a feature vector. Using these features I trained a neural network to predict the probability that A > B for any pair ...
2
votes
2answers
59 views

Why don't people use deeper RBFs or RBF in combination with MLP?

So when looking at Radial Basis Function Neural Networks, I've noticed that people only ever recommend the usage of 1 hidden layer, whereas with multilayer perceptron neural networks more layers is ...
0
votes
1answer
35 views

Backward propagation algorithm demonstration in neural networks: any VERY-SMALL-STEP by VERY-SMALL-STEP demonstration?

I'm looking for a VERY DETAILED demonstration for the backward propagation algorithm in neural networks machine learning. Specifically the step below. I've got the excellent Michael Nielsen ...
0
votes
0answers
27 views

Trying to fit single layer neural net with R's nls (nonlinear least squares) function

Working on building a neural network modeling frame using graph objects in R. I have a data set on passengers of the Titanic, modeling binary "survived" variable against continuous "fare" and "age" ...
1
vote
0answers
25 views

Neural Network training with Levenberg Marquardt without validation set

I wish to implement a matlab version of the well-known Levenberg-Marquardt algorithm to train a given set of data. The size of the available data is small - hence, making the use of cross validation ...
0
votes
0answers
24 views

How to input sparse feature

In theano everything is symbolic, so how to input sparse feature in , for example, neural network? The setting is: the task is text application. the input is a mini-batch. Since theano sparse module ...
0
votes
0answers
16 views

Radial Basis Function Networks for Classification

I'm thinking of implementing a radial basis function network for a multinomial classification problem. Is there any benefit to this over using gaussian mixture modeling? Are they essentially the same ...
0
votes
0answers
13 views

What do performance and gradient parameter signify in nntraintool progress?

I am using Pattern Recognition Tool for Image Classification. nntraintool uses trainscg algorithm. I am not able to understand the parameter values displayed under progress bar. Attaching the same. ...
0
votes
1answer
39 views

Robust softmax solutions for Theano?

I am implementing multilayer perceptrons with the softmax activation function over Theano. In some extreme cases I am running into problems with too high/low values in the softmax function that ...
0
votes
0answers
30 views

Difference between regression and performance plot of Artificial neural network in MATLAB

I am having problem understanding regression and performance plots of ANN. My data consists of 13 inputs and 3 outputs. Parameters used for simulation can be found here. The problem I am facing is ...
-1
votes
1answer
48 views

Vectorization of Cross Entropy Loss

I am dealing with a problem related to finding the gradient of the Cross entropy loss function w.r.t. the parameter $\theta$ where: $CE(\theta) = -\sum\nolimits_{i}{y_i*log({\hat{y}_{i}})}$ Where, ...
1
vote
1answer
59 views

MLP: Classification vs. Regression

Abstract I am teaching myself about NNs for a summer research project by following an MLP tutorial which classifies the MNIST handwriting database. I want to change the MLP from classification to ...
0
votes
0answers
37 views

Choosing the parameters for an artificial neural network for time-series regression in R

I'm trying to build an artificial neural network (ANN) using the R "neuralnet" package, to predict streamflow from snow albedo (reflectance of the snow; controls the amount of heat absorbed by the ...
0
votes
0answers
13 views

Neural networks kernel for high error rate in training set.

I will be working with a huge training set (around 10^6 examples, with around 400 features). Which has labels (around 100) accurate to around 90%. It would be possible to generate a smaller subset of ...
0
votes
2answers
37 views

What differentiates one feature map from another in CNN

I understand in a convolution neural net that you may have several feature maps in the same layer, for instance one map detects curly loops for some letter and another detects straight lines. Here my ...
0
votes
0answers
34 views

Tuning a neural network

I have been designing a neural network to perform predictions on construction item costs. I've developed a core set of predictors that seem to describe the problem space well - they appear to be ...
0
votes
0answers
34 views

neural network multiple layers feed forward

NN on figure below has two nodes (N0,0 and N0,1) in input layer, two nodes in hidden layer (N1,0 and N1,1) and one node in output layer (N2,0). Input layer nodes are connected to hidden layer nodes ...
0
votes
0answers
23 views

Which one is correct phase for neural network or support vector machine? Features or Inputs?

Which one is correct phase for Neural Network or Support Vector Machine? Features or Inputs? Based on ...
1
vote
0answers
20 views

Relative advantages of nnet, neuralnet, caret and RSNNS packages

What are the relative advantages and disadvantages of different packages available for neural networks: nnet, neuralnet, caret and RSNNS? Which is best in terms of simplicity? Which is best for ...
0
votes
1answer
22 views

Pros of back propagation learning algorithm

I'm doing some research into neural networks and it seems like every single one i've come across implements a back-propagation algorithm. Is this because they're very easy to implement or are there ...
7
votes
0answers
62 views

Rules for selecting convolutional neural network parameters

Are there any good papers that cover some methodical ways of picking dimensions of filters and pooling units as well as the number of convolutional layers?
-1
votes
1answer
32 views

Somebody explain Training, Testing and Validation Test of Artificial Neural Network [duplicate]

What is the procedure of Training, Testing and Validation Test? Explain it thoroughly. Or give some link for related articles
0
votes
0answers
17 views

How can I calculate the AUC for softmax classifier (e.g., logistic regression)?

At the end of a convolutional neural network(CNN) , there are usually a softmax classifier attached to it. How can I calculate the AUC for the CNN (that is, for the softmax classifier)? Thanks!
0
votes
1answer
41 views

Neural Network Forward Propagation

I'm trying to solve this neural network problem found here: How do I go ahead and calculate the forward propogate in this example? I've see examples of how to calculate the expected output but ...
1
vote
0answers
9 views

Implementing a Radial Basis Function Network. Question about missing information

I would like to implement a Radial Basis Function (Neural) Network. Specifically, I would like to implement the network as described in this paper: http://www.ncbi.nlm.nih.gov/pubmed/15732389. The ...
0
votes
1answer
51 views

Which machine learning model is applicable to the following case

I want to build a model that recognizes the species based on multiple indicators. The problem is, neural networks (usually) receive vectors, and my indicators are not always easily expressed in ...
0
votes
0answers
7 views

Guarded Discrete Stochastic network to solve CSP

There is a lot about NPL and vision systems but not a lot about constraint satisfaction problems (CSP). I have been digging around but I could use some guidance on how to use neutral network to solve ...
0
votes
1answer
45 views

What is pretraining and how do you pretrain a neural network?

I understand that pretraining is used to avoid some of the issues with conventional training. If I use backpropagation with, say an autoencoder, I know I'm going to run into time issues because ...
0
votes
0answers
23 views

Overfitting in the validation set

When running an algorithm for training a system it is common to consider a lot of models and using the validation set for selecting one of them. In my case I am running a mini-batch gradient descent ...
0
votes
0answers
12 views

Prove Reccurrent Neural Network can exhibit oscillatory behavior

I understand how recurrent neural networks work, however I'm trying to build a deep intuitive understanding of their behavior which is difficult for me because they exhibit such complex behaviors. ...
4
votes
1answer
43 views

Regularization in Neural networks

One way to regularize a neural network is "early stopping" , meaning that I don't let the weights get to their optimal values (based on the cost function calculated on the training data) but stop the ...
0
votes
1answer
54 views

What are the best R packages for a classification problem with use of Neural networks [closed]

Surfing on the internet shows me that there are a lot of different packages and functions which can be used to train neural networks via R. packages such as 'RSNNS', 'nnet','neuralnet', etc. I'm ...