Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
0answers
13 views

Representation of misspelled words for neural network?

While thinking about a neural network based spellchecker, I was thinking about word embedding not being able to represent any "unique" (misspelled) words that the model haven't seen before. I tried ...
0
votes
0answers
13 views

Is there are better way of solving this problem than using a Neural Network Regression Model?

I'm working on power plant time series data from a SO2 absorption process and my main objective is finding out which independent variables are critical for reducing SAG (% of SO2 concentration ...
0
votes
0answers
12 views

How does local connection implied in the CNN algorithm

I am trying to understand the process of Convolutional Neural Networks. Basically, I am trying to understand how does the local connection works. The first step of CNN is a convolution layer where ...
0
votes
0answers
8 views

Does it ever make sense to use a feature-local (e.g. polynomial) kernel for binary data?

As I understand, for a sample $s$, a polynomial kernel produces a vector consisting of $x_{s,i},x_{s,i}^2,..., x_{s,i}^n$ for every feature $i$, allowing SVM (or ANN) to effectively find a nonlinear ...
0
votes
0answers
20 views

Getting an error message “Error in if(reached.threshold < min.reached.threshold)…” while training network using neuralnet package

I'm using R to create train and test a neural network on a time series (the annual sales of a company over a large period of time). As using the package's default learning algorithm (resilient ...
1
vote
1answer
72 views

Using RNN (LSTM) for predicting the timeseries vectors (Theano)

I have very simple problem but I cannot find a right tool to solve it. I have some sequence of vectors of the same length. Now I would like to train LSTM RNN on train sample of these sequences and ...
10
votes
3answers
479 views

What can we learn about the human brain from artificial neural networks?

I know my question/title is not very specific, so I will try to clearify it: Artificial neural networks have relatively strict designs. Of course, generally, they are influenced by biology and try to ...
1
vote
1answer
32 views

What is a feasible sequence length for an RNN to model?

I'm looking into using a LSTM (long short-term memory) version of a recurrent neural network (RNN) for modeling timeseries data. As the sequence length of the data increases, the complexity of the ...
1
vote
0answers
19 views

How to deal with different sizes of sentences when giving them as input to a Neural Network?

I am giving a sentence as input to a tree structured Neural Network, where the leaf nodes will be the word vectors of the words in the sentence. That tree will be a binarized constituency(see the ...
2
votes
1answer
44 views

train an SVM via back propagation?

I was wondering if it was possible to train an SVM (say a linear one, to make things easy) using back propagation? Currently, I'm at a road block, because I can only think about writing the ...
0
votes
0answers
18 views

Advantages of the softmax function in feedforward multi-class neural nets over logistic activation and one vs all approach

I am wondering if there is a benefit to the softmax function over an one-vs-all sigmoid activation function approach in feedforward neural networks for multi-class classification -- except for the ...
2
votes
3answers
57 views

Use Edge detection in Image classification

I am having five types of objects (flower, building, face, pair of shoes and a car) in my object recognition and i need to classify these. Identifying through edges in this type of data set seems to ...
4
votes
1answer
43 views

Why are activation functions needed in neural networks?

Why are activation functions needed in neural networks? I know that it is to capture "non-linearities", but I have never been able to find a proper down-to-earth explanation. In particular, I am ...
0
votes
0answers
20 views

Conclusion from PCA of dataset

I have a set of data for sequence labeling. I did PCA with (with 2 principal components on the x and y axis) on the dataset and it turns out as below: Using an LSTM network to classify the dataset ...
2
votes
0answers
39 views

How are radial basis functions (RBFs) networks extended to use multiple layers?

I am trying to understand the interpretation of radial basis functions (RBFs) as networks and then trying to understand the relationship it has to "normal" neural networks and how to extend them to ...
3
votes
2answers
115 views

Can a neural network learn a functional, and its functional derivative?

I understand that neural networks (NNs) can be considered universal approximators to both functions and their derivatives, under certain assumptions (on both the network and the function to ...
3
votes
1answer
44 views

How do gradients propagate in an unrolled recurrent neural network?

I'm trying to understand how rnn's can be used to predict sequences by working through a simple example. Here is my simple network, consisting of one input, one hidden neuron, and one output: The ...
0
votes
0answers
19 views

reason codes for non-linear models?

I have a non-linear model with n variables (ANN model). The variables are WOE-transformed to train the model. I have a test record scored using the non-linear model mentioned above and it is in the ...
0
votes
3answers
47 views

Basic ANN questions

So I've just recently gotten into artificial neural networks, and I have a couple of questions that I can't seem to find addressed anywhere. Firstly, this one is more specific to image recognition, ...
1
vote
2answers
65 views

Is it right to consider the output of the neural network as its confidence in predicting the output?

Suppose I have a single output sigmoid (tanh) that is producing an output ranging [-1, +1]. Is it right to consider this output as its confidence measue of predicting the output. The output value ...
0
votes
1answer
17 views

Early stopping for CNN to improve speed of training

I want to implement early stopping for my convolutional neural network. The main reason is that I want to test my CNN using various parameter settings and some of these may require more iterations ...
1
vote
1answer
28 views

How can I do a one vs all classification (binary classifier) with a neural network

I have a set of images that belong to a particular class. Then, I have another set of images that do not contain any image of the above particular class. So, effectively I have two sets of images ...
0
votes
1answer
35 views

Can neural network classify large images?

I'm considering using ReLU or convolutional deep learning network to classify black and white 8.5"x11" images (with some fine details). Most examples of DNN I saw tested on the MNIST images which are ...
0
votes
0answers
10 views

Having trouble understanding/implementing backpropagation algorithm

I have a simple feedforward neural network with 2 input neurons (and 1 bias neuron), 4 hidden neurons (and 1 bias neuron), and one output neuron. The feedforward mechanism seems to be working fine, ...
3
votes
1answer
121 views

Neural network Equation question

I am looking at an example for the activation function $a_1$. Why does the equation look like $\Theta_{10}x_0 + \Theta_{11}x_1 + \Theta_{12}x_2 + \Theta_{13}x_3$ instead of $\Theta_{10}x_0 + ...
2
votes
1answer
44 views

Neural network - binary vs discrete / continuous input

Are there any good reasons for preferring binary values (0/1) over discrete or continuous normalized values, e.g. (1;3), as inputs for a feedforward network for all input nodes (with or without ...
1
vote
2answers
43 views

Is a neural network required to perform with very high accuracy in the training data itself?

I am training a binary classifier and at the end of every epoch I am running the trained network on the training data itself again. Is it important to get a very high accuracy on the above step at ...
0
votes
0answers
48 views

Neural network learns different functions, depending on the training method

I am trying to train a basic feedforward neural network to learn the function $f(x,y) = y^3$ (note that the function purposely doesn't actually depend on x). Depending on the training method, however, ...
1
vote
1answer
32 views

Positive or negative effect of neural network inputs on output in binary classification (MATLAB)?

How we can find an input has positive or negative effects on output in a binary classification neural network in MATLAB R2015a? (with ...
0
votes
1answer
30 views

Getting a constant answer while finding patterns with neuralnet package

I'm trying to find patterns in a large dataset using the neuralnet package. My data file looks something like this (30,204,447 rows) : ...
-1
votes
0answers
11 views

Activation function of the output layer in a Neural Network

I am trying to design a neural network to predict a sin function. I am using sigmoid for the activation function of the nodes. ...
0
votes
0answers
28 views

What is the activation function, label and loss function for Hierachical Softmax [migrated]

Several papers([1],[2], [3]) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get clear in ...
2
votes
0answers
50 views

Back-prop question: can this gradient be decomposed?

So, I was going over the lectures for the Oxford 2015 deep learning course, and in the lectures, they introduce back-propagation as a recursive procedure which involves two key formulas: The ...
0
votes
1answer
66 views

Convolutional Neural Network process

I have two question regarding regarding Convolutional Neural Network (with Autoencoders for patches generation). Let's assume that I got a dataset with images and I want to perform an object ...
0
votes
1answer
38 views

Which features are most relevant to each class in neural for network binary classification?

I designed a neural network for binary classification in MATLAB R2015a. What are differences between two classes? How system detects a sample is from class 1 or ...
5
votes
1answer
58 views

Choosing a method to solve a many-to-one mapping problem

Problem description To predict a list of values associated with a set of variables. Trainset Trainset has a set of variables X1, X2, X3, ... Xn. In the simplest form, each variable is of type ...
2
votes
0answers
29 views

Autoencoder doesn't work (can't learn features)

I am completely new to machine learning and am playing around with the theanets package. What I am currently trying to do is to get an Autoencoder to reproduce a series of Gaussian distributions: ...
2
votes
1answer
50 views

How to derive the recursive equation for back propagation for neural networks for $\delta_j = \frac{\partial E_n}{ \partial a_j} $

I am following the derivation for back propagation presented in Bishop's book Pattern Recognition and Machine Learning and had some confusions in following the derivation presented in section 5.3.1. ...
0
votes
1answer
37 views

The derivation of $\delta_j = \frac{\partial E_n}{ \partial a_j}$ errors for hidden units in back propagation for neural networks with the chain rule

I was trying to understand the derivation for back propagation for multi-layer neural networks from Bishop's Pattern Recognition and Machine Learning book. Specifically I was reading section 5.3.1 ...
2
votes
1answer
44 views

How is convolutional network used to locate logos in images?

I have a large set of logos (think of it as kind of logos of automobile companies). Now, I want to train a convolutional network to locate the logo in a given image. Are there any papers that talk ...
0
votes
2answers
87 views

Siamese neural network, why is my gradient descent updating in the wrong direction?

I've been trying to implement a siamese neural network in Torch/Lua, as I already explained here. Now I have my first implementation, that I suppose to be good. Unfortunately, I'm facing a problem: ...
0
votes
1answer
25 views

Building an image classifier( binary) using Convolution network network

I have a dataset of images of birds and want to build a CNN classifier that outputs the probability that the fed image(test) is a bird, So that I can accept the image to be a bird beyond a certain ...
0
votes
0answers
16 views

What could cause unreproducible neural network training performance?

I'm trying to select various hyper-parameters, especially hidden layer sizes and depth, for my back-propagation ReLU neural network. I define a procedure to train/test a given configuration by ...
1
vote
0answers
37 views

How to normalize mixed continuous/discrete features for DNN?

I have had some success training my deep neural network (with ReLU hidden units) by first normalizing the features of my data set to zero-mean-unit-variance. Each sample of my data set has 600+ ...
6
votes
1answer
54 views

What does VC dimension tell us about deep learning?

In basic machine learning we are taught the following "rules of thumb": a) the size of your data should be at least 10 times the size of the VC dimension of your hypothesis set b) a neural network ...
1
vote
2answers
50 views

What does the notation $t_{nk}$ mean for neural networks in Bishop's Pattern Recognition book?

I was reading Bishop's Pattern Recognition book, specifically, I was reading his notation for expressing the error just before back propagation. The particular equation I am a little confused about is ...
0
votes
0answers
8 views

Using a neural network for prediction

I am new to neural network and want to see is it a valid approach for predicting data points that are repeatable in time? For example, the sequence looks like ...
0
votes
1answer
87 views

Why does my neural net fail to learn higher frequency sine waves?

I am testing my neural network implementation. I have an input layer with a single unit, one hidden layer consisting of 65 tanh units, and an output layer ...
0
votes
1answer
23 views

What are the techniques used for learning in non-feedforward neural networks?

Suppose our network architecture has a hidden layer in which the hidden units are interconnected, then is there some sort of variation on backpropagation that is used? What about in general recurrent ...
0
votes
1answer
19 views

Why do we not care about target class probabilities of 0 in the cross entropy equation?

Only the target classes where probability is equal to 1 contributes to the loss. I'm dealing specifically with neural networks, but the question is general.