Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
0answers
12 views

Sum Product Network

Not sure if this is the place to ask this question. Been trying to google and understand what is sum product networks but it seems to difficult for me to understand. Does anyone have any links or ...
0
votes
1answer
12 views

Neural network using Matlab [on hold]

I have one question which is :I have 2-Dimensional data ,so i want insert that data to neural network tool in matlab and what is the process to insert 2-D to neural network using matlab ,Thank you for ...
2
votes
1answer
31 views

Classification with partially “unknown” data

Suppose I want to learn a classifier that takes a vector of numbers as input, and gives a class label as output. My training data consists of a large number of input-output pairs. However, when I ...
1
vote
0answers
25 views

Using gradient information in minimizing error function, in Bishop's Pattern Recognition

In Bishop's book Pattern Recognition, there appears the following paragraph on page 239, where I included the equation he refers to In the quadratic approximation to the error function, ...
1
vote
0answers
15 views

Detect multiple classes in an image?

I have a deep neural network trained with data of different kinds of fruits (apples, oranges, guava, pear, etc.). In my testing data, I have multiple fruits in the same image. For example, an image ...
0
votes
1answer
20 views

sigmoid threshold for neural network

I've trained a NN for a few logical linear logical functions. I've used the sign(positive=1, negative=0) function to threshold the hypothesis. (That makes a Perceptron right?) Now I want to use the ...
0
votes
1answer
17 views

Plotting error vs weights for neural networks

What is the better estimate of how well your NN is doing? Plotting how weights change with the number of iterations or Plotting how the error changes with the number of iterations ?!
0
votes
0answers
10 views

Why my Convolutional Neural Network always produces the same outputs?

I used MatConvNet to build a CNN model for regression. The input size is 20×20×1×32, the output size is 4×1×32, the convolutional filter size is 3×3×1. Now I found after training the training error ...
0
votes
0answers
14 views

Is cross entropy only applied on the last layer of an ANN?

I was reading about L1 regularization and from what I understand, we compute the cost of the last layer like: $\ w_{new} = w−\eta\frac{\partial C_{0}}{∂w}−\frac{\eta\lambda}{n}w $ However when using ...
1
vote
0answers
11 views

Compute all paths in graph that has multiple inputs and one output

I want to compute all the paths in directed acyclic graph from multiple inputs (x1, .., xn) to one output. The graph has the same depth which d and the inputs come to the graph at the same time (the ...
0
votes
0answers
15 views

Proposal Topic for solving a certain robotics problem by combing Deep Reinforcement Learning and Computational Neuroscience together [on hold]

Due to current master degree curriculum (7 courses in total: 1 in AI, 4 in Machine Learning including Reinforcement Learning and Deep Learning, 2 in Probabilistic Robotics) plus a module in ...
0
votes
0answers
17 views

Deep Recurrent Neural Net (RNN) implementation/toolbox in MATLAB [on hold]

Could you please advise me about any Deep Recurrent Neural Net (RNN) implementation/toolbox in MATLAB (using pre-training and fine-tuning). Thanks
0
votes
0answers
11 views

How do you train neural network to consider context?

I'm trying to train a neural network to be able to distinguish context in data. If I have a generated input such as in http://postimg.org/image/gahwhqcct/: I would like to train it give an output ...
1
vote
1answer
26 views

How to plot the OR function along with the decision boundary of a Perceptron?

I've written a small program that predicts correctly the OR function output. The problem is that when I try to plot the decision boundary, I don't know what to do. Should I plot the final weights?. ...
1
vote
1answer
24 views

Bias inputs in an RNN

As far as I'm aware, the bias inputs for a feed forward neural network are typically connected as follows: How are they connected in a recurrent neural network? (My guess is below)
0
votes
0answers
18 views

What neural network architecture and weights can approximate the following functions?

I learned how to represent all the boolean functions such as AND, OR, XOR etc. If a multilayer (recurrent) neural network can approximate any function arbitrarily close, what kind of architecture and ...
0
votes
0answers
23 views

Cost function spiking upon using dropout on neural network

Upon using the dropout technique, my cost function is spiking arbitrarily. Is this normal? If not, how do I avoid it? I'm using a salt-and-pepper mask to drop out neurons at a dropout rate of 5%. ...
0
votes
2answers
37 views

Why it is popular to use stochastic gradient descent in neural networks rather than the BFGS algorithm?

I have made two solvers to implement neural networks, one is based on stochastic gradient descent (SGD) while the other is based on the BFGS (Broyden-Fletcher-Goldfarb-Shanno) algorithm. I have read ...
0
votes
0answers
24 views

sentiment analysis using convolutional neural networks

I was trying to modify YoonKim's code for sentiment analysis using CNN's. He applies three filters of heights=[3,4,5] and ...
0
votes
0answers
30 views

Plotting neural net decision boundaries

I'm trying to plot the decision boundaries created by a 3 layer, feedforward neural net over the input data. The network has 5 nodes in the input layer (including a bias node), 6 nodes in the hidden ...
0
votes
1answer
37 views

Why normalize input variables in NN?

I'm reading the 'Efficient Backprop' paper and it's mentioned that the reason to have a zero mean for the input variables is because otherwise the eigenvalue (for the hessian I think) will be very ...
0
votes
0answers
14 views

Changing optimization algorithm while optimizing

I am currently training some convolutional neural networks with cross-entropy loss. Thus, the function I am optimizing is non-convex, and at the moment I am using an optimization algorithm called Adam ...
1
vote
0answers
17 views

What optimization methods work best for LSTMs?

I've been using theano to experiment with LSTMs, and was wondering what optimization methods (SGD, Adagrad, Adadelta, RMSprop, Adam, etc) work best for LSTMs? Are there any research papers on this ...
1
vote
0answers
26 views

What is the neural net input/output architecture for generic object detection?

I am trying to understand this paper: http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Erhan_Scalable_Object_Detection_2014_CVPR_paper.pdf They worked on object detection without ...
0
votes
0answers
10 views

How to add extra layer of MLP to DBN

I am trying to add MLP layer to DBN that can use final parameters of DBN model as Input for MLP model. I am new to python so am not well versed with its input and output processes. Any help is ...
0
votes
0answers
8 views

depth column in convolution neural net

I am trying to understand intuitively the usefulness of organizing the convolution layer with depth columns. As I understand it each depth column is the convolution of the same section of the input ...
0
votes
0answers
53 views
6
votes
2answers
414 views

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by ...
0
votes
0answers
8 views

Backpropagation with Cross-entropy Cost Function

I'm using the cross-entropy cost function for backpropagation in a neutral network as it is discussed here: ...
1
vote
1answer
26 views

Cross-entropy Cost Function in Neural Network

I'm looking at the cost function found here: http://neuralnetworksanddeeplearning.com/chap3.html#introducing_the_cross-entropy_cost_function What are we summing over in: C= −1/n ∑x ...
1
vote
1answer
64 views

Understanding k-means unsupervised learning for features

I'm following this paper: http://ai.stanford.edu/~ang/papers/icdar01-TextRecognitionUnsupervisedFeatureLearning.pdf And I'm trying to understand specifically how the k-means approach works when ...
1
vote
1answer
25 views

Capturing initial patterns when using truncated backpropagation through time (RNN/LSTM)

Say that I use an RNN/LSTM to do sentiment analysis, which is a many-to-one approach (see this blog). The network is trained through a truncated backpropagation through time (BPTT), where the network ...
0
votes
0answers
16 views

What is the difference between Sigmoid neurons and Stochastic binary neurons?

Both have the same equation : the logistic unit. Sigmoid output a ral-valued number between 0 and 1 and Stochastic binary neuron a probability between 0 and 1 too. Apart from the name/type given to ...
1
vote
0answers
17 views

No need for standardization with Adadelta? (RNN/LSTM)

Often it is best to standardize data before inputting it into a machine learning algorithm. This is also the case with deep learning algorithms such as convolutional neural network. However, when ...
0
votes
0answers
58 views

How to deal with imbalance data in , for example, neural network

Do we usually discard this issue, just train the neural network then compute the AUC, or can we use weighted version of loss function, for example, in binary classification problem, can we use ...
0
votes
0answers
22 views

Back Propagation Algorithm Check

I was a little confused on the back propagation algorithm for neural networks and was hoping someone could verify my understanding. 1) After running the inputs forward through the network find ...
0
votes
1answer
28 views

What statistical test should I run to select “explicative” features in my dataset?

I have a database with more than 500 samples with 22 quantitative features each and I would like to predict a categorical variable (0 or 1). I am trying to fit a logistic regression model and a neural ...
0
votes
0answers
36 views

Choosing contrast coding in R

I am working on a data set with categorical variables. To apply ANN, I want to apply contrast coding to those variables. But how do I choose between coding functions in R (sum, helmert, treatment ...
2
votes
1answer
18 views

Cost Function (Neural Network) for Forward Propagation

This question is related to Andrew Ng's machine learning course on Coursera. Basically, when I calculate the cost function of a neural network, I use the following formula that was described by Ng: $$ ...
1
vote
0answers
21 views

How effective is randomly/algorithmically generating exorbitant amounts of training data for a neural network?

There are some problems of which the generating of data is easy whereas the inverse is not. For example, use a 3D game engine to render some randomly generated objects with some random changes and ...
1
vote
2answers
45 views

In back propagation for neural networks, what exactly is the “error signal”?

For example: Imagine we end up with a sum of 0.755 on our output node. We then apply the activation function (in this case I'll use a sigmoid) which gives us a ...
0
votes
1answer
17 views

Are 'neurons' and 'feature detectors' the same? (in the context of neural networks)

Are 'neurons' and 'feature detectors' the same thing in the context of neural networks? Are neurons feature detectors? Or is there some fundamental difference that I'm missing?
3
votes
0answers
61 views

How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...
1
vote
1answer
24 views

Under-fitting neural net for regression

I have ~30,000 instances and 500 dimensions. My target variable is continuous, so I'm preforming regression with no (i.e. linear) activation at the output. I'm unable to fit the data, as my training ...
0
votes
0answers
18 views

Estimating importance of variables in a multilayer perceptron

From online search so far, I have only found Garson Algorithm as a method for deducing the importance of variables in a Multilayer Perceptron. However the current Garson algorithm included in the ...
0
votes
1answer
32 views

MNIST dataset black or white background

In the MNIST dataset, are the images on white or black background? I seem to have encounter both type of images by googling around. Does the background color has any effects on the performance of a ...
2
votes
1answer
42 views

How to apply Cross Entropy on Rectified Linear Units

I am currently getting started with Machine Learning. However, I have some problem to derive formula and not able understand how to applied the Cross Entropy (CE) on Rectified Linear Units (ReLU). I ...
0
votes
0answers
30 views

Neural Net in high dimensions for images

I'm trying to build a neural net for a image recognition problem. My images are way too large to build a straight up NN from just the pixels; they are about (1000, 1000) width,height. So naturally i'm ...
0
votes
0answers
31 views

Neural network performance differs and depends on number of input and output arguments

Have been experimenting with scaled conjugate gradient (SCG) backpropagation with 1 hidden layer. Once added more input parameters, the perfomance became worse both on training and testing data sets ...
4
votes
1answer
185 views

Correct way of handling the output layer deltas in neural networks

According to Andrew Ng's Coursera Machine Learning course (notes here and here), a training algorithm for neural nets is this: $\delta^{(L)} = a^{(L)} - y$ $\text{for }l = L-1, ..., 2\text{ do}$ ...