Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

1
vote
0answers
16 views

What Percent of Neural Network is used while processing a single image

What percent (on average) of entire Neural network (say, AlexNet) is actually used while processing a single image. There should only a very small amount of network that should actually be utilized ...
0
votes
0answers
7 views

Pre-training in deep convolutional neural network?

Have anyone seen any literature on pre-training in deep convolutional neural network? I have only seen unsupervised pre-training in autoencoder or restrcited boltzman machines.
0
votes
0answers
23 views

How do you turn the output of a nnet neural network model into an equation?

Assuming the output of the above nnet feedforward model (nnetModel) is such that the following summary is produced: ...
0
votes
0answers
11 views

Which data transformation can improve the performance of MLP neural networks for classification?

I am trying to fit several MLP neural networks models with a single hidden layer using the caret R-package. My main concern now is in the preprocessing step. My train data features (16 in total) are ...
0
votes
1answer
11 views

Single artificial neuron easily extendable to neural network

I'm working on implementation of artificial neuron which be extended to neural net. I want do implementation by myself to fully understand how it works. I start with perceptron with threshold ...
1
vote
0answers
10 views

Advantages of using multiple lstm s in deep network

What are the advantages, why would one use multiple lstm s, stacked one side-by-side, in a deep-network? I am using a lstm to represent a sequence of inputs as a single input. So once I have that ...
0
votes
0answers
7 views

What are word components in IAM db3 for xml

I'm using the IAM handwritten database and for the answer keys I get the hierarchy, however I'm confused about word components in the the handwritten section. Specifically seeing something like this ...
0
votes
0answers
60 views

Is it possible to use genetic algorithms as a learning algorithm for artificial neural networks?

I've been using back-propagation to optimize neural networks without problems. But I've read in some books that genetic algorithms can be used to optimize an ANN. I want to know if its possible to use ...
1
vote
0answers
20 views

Derivation of formulas for Boltzmann machines - MCMC

With interest i read the latest post on https://theclevermachine.wordpress.com/ on Boltzmann machines, and the derivation of the underlying formulas. The derivation (per below) shows that the ...
0
votes
0answers
12 views

Problem with data on Matlab Neural Network Toolbox

I am currently working on a NARX network for a time-series prediction problem. I am using a 3*4644 array as inputs and a 1*4644 array as my targets. I do not have any delay on the input and the ...
0
votes
1answer
24 views

What are the differences between delta rule and generalized delta rule?

I know that the delta rule is a gradient decent learning rule. But, what are the differences between these two delta rules? Thanks in advance.
0
votes
1answer
26 views

Leave one out cross validation for neural network perfomance

When using leave one out cross validation in neural network, do I have to fix the epoch number for each training model? The test results of these models are averaged to show performance. So can I ...
2
votes
1answer
62 views

How to set the dictionary for text analysis using neural networks

I want to use a neural network to do text analysis. If I use a large dictionary, then it will contain all the words in training and test set, but the size of the dictionary is too large which will ...
9
votes
3answers
169 views

Why sigmoid function instead of anything else?

Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression? Why don't we use many of the other derivable functions, with ...
2
votes
2answers
25 views

Continue the predictions beyond the current data using time series neural network

I have a single time series variable and I want to train a neural network in a sort of auto-regressive fashion. specifically, I have data for water consumption that is changing with time In the ...
0
votes
0answers
9 views

How to use Galgo for variable selection. [closed]

I would like to make use of genetic algorithm implementation in R GALGO for variable selection and the result will be fed into multilayer neural network for classification.I do not know how to go ...
1
vote
1answer
27 views

Neural network to tag multiple textual topics in a single document

I want to use a neural network to do some topic analysis in a textual corpus. I have used neural networks before where there is a clear decision boundary between the category to which some observation ...
0
votes
0answers
43 views

Train neural network for forecasting

I am trying to use time series neural network to predict future values. I have time series data from 2010-2014 and I need to predict the values from 2015-2020 using time series neural network. I am ...
4
votes
1answer
53 views

Why doesn't deep learning work well with small amount of data?

I am new to deep learning, so this might be a trivial question. But I am wondering why deep learning (or neural network) does not work very well on small labeled data. Whatever research papers I have ...
1
vote
1answer
15 views

Training lstm a sequence one item at a time

I am trying to train an lstm with a sequence and get the sequence classification for the whole sequence. I have sequences of varying length so I have one input neuron and I am feeding one item at a ...
0
votes
1answer
21 views

About implementing convolutional neural network

I want to implement a convolutional neural network using the tiny-cnn implmentation for c++. I have downloaded it and tried the MNIST example in there, but I'm having trouble implementing it for my ...
0
votes
0answers
12 views

Time series forecasting using ANN

I have an array of data recorded from vibration analysis of a bearing.I want to know how to forecast 30 day later. I don't know machine learning and I'm not so familiar with neural network for example ...
1
vote
0answers
56 views

What are key differences between Theano (Python) and Torch (Lua) for deep learning? [closed]

Theano and Torch both supports GPU calculations. My question is whether Theano or Torch have significant differences in: performance ease of use (assuming one knows the programming language) libaray ...
0
votes
0answers
13 views

multidimensional inputs, outputs and backpropagation

Let's say I have a neural network in matrix form. Inputs, hidden layer nodes and outputs are represented by row vectors, while the weights are matrices of the sizes [outputRows; inputRows]. Now, let's ...
0
votes
0answers
31 views

Personal bet suggestor

Hi which regression or classification algorithm gives the best suggestions for future bets. i have a small training data base (approximately 50 data) consists my old BET coupons and old results. And ...
0
votes
1answer
26 views

Training set size for neural networks considering curse of dimensionality

I'm learning the ropes of neural networks. Recently, I read stuff about the curse of dimensionality and how it might lead to overfitting (e.g. here). If I understand correctly, the number of ...
0
votes
0answers
25 views

Information retrieval from strings using neural network

I am dealing with input strings that have max length of 200 characters, they are the USSD popup msgs and the SMS text strings on android smart phones extracted as and when they appear via an android ...
1
vote
0answers
45 views

Bayesian neural networks: very multimodal posterior?

Question: How do Bayesian treatments of neural networks address the fact that the posterior has an exponentially large number of modes? Background: There seems to be a lot of interest in Bayesian ...
1
vote
1answer
20 views

Quadratic error for multi-class classification

I'm trying to train a neural network to classify handwritten inputs into 10 categories, each for one digit (1,...,9,0). I represent the output of an example using a 10-dimensional vector. Digit 5, for ...
0
votes
0answers
33 views

Problem with backpropagation algorithm for Feedforward Neural Networks

The objective When trying to exercise my knowledge of Feedforward Neural Networks, I started implementing one. The result is here. The final goal is to predict some handwritten digits data I have. ...
1
vote
1answer
27 views

Need guidance in image classification

I'm new to machine learning and need some help. I need image classification to tell if an image is a car or not. Is there any working example or guidance or a book for this particular question? ...
1
vote
0answers
26 views

What is a compact vector equation expression the back-propagation algorithm for convolution neural networks?

I was reading the lecture notes for sparse auto-encoders from Andrew Ng and was saw that it had very nice compact way of expressing back propagation for neural networks: The really nice thing about ...
1
vote
0answers
12 views

Training A Restricted Boltzmann Machine with gray-scale images

I am trying to train a feed forward neural net for image classification. In this process, I am implementing a restricted boltzmann machine to help pre-train the weights. My images are grey-scale ...
0
votes
1answer
67 views

Train a Neural Network to distinguish between even and odd numbers

Question: is it possible to train a NN to distinguish between odd and even numbers only using as input the numbers themselves? I have the following dataset: ...
0
votes
0answers
24 views

Neural Network poor training performance

I'm currently trying to train a neural network for regression. The function the network should approximate is a sinus function from a 3 dimensional real input to a one dimensional real output. The ...
0
votes
1answer
42 views

What does the equation $h^k = \sigma(x * W^K + b^k)$ mean in the context of convolutional neural networks (CNNs)?

I was reading a paper on CNN for auto-encoders and in section 3 they had the following section: For a mono-channel input $x$ the latent representation of the k-th feature map is given by $$ h^k ...
2
votes
2answers
76 views

How exactly do convolutional neural networks use convolution in place of matrix multiplication?

I was reading Yoshua Bengio's Book on deep learning and it says on page 224: Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at ...
1
vote
0answers
33 views

Why do auto-encoders with 1 hidden layer usually use the output weights/filter as $W=W^T$?

I was trying to understand why for auto-encoders with 1 hidden layer, we usually use the output weights/filter as $W=W^T$. Is there any theoretical justification to use $W=W^T$? Or maybe any way to ...
0
votes
1answer
13 views

In the context of ANNs, is there multi-task learning iff the network has more than 1 output?

This is a terminology question: in the context of artificial neural networks, does multi-task learning occur iff the network has more than 1 output?
0
votes
0answers
12 views

Finding keywords in small pieces of text.

I'm (attempting to) construct a bayesian neural network to parse through tweets to then decide if someone bought/sold something and for how much. I'm having trouble figuring out what the input nodes ...
0
votes
1answer
29 views

Number of nodes in hidden layers of neural network

I have a neural network with 3 hidden layers and I'm unsure about the number of hidden nodes for each layer. Should the number of hidden nodes stay constant between the hidden layers, e.g. 500 nodes ...
0
votes
0answers
14 views

How can I select the best combination of parameters for this neural network?

I have tabulated the following learning results for a 3 layer neural network. The results are obtained with different combination of [number of iterations/regularisation parameter lambda/number of ...
0
votes
0answers
7 views

Are multinomial logits modelled by multinom() in {nnet} robust to the linearity assumption?

I am modelling a multinomial logit using the multinom() function in the nnet package: fit <- multinom(outcome ~ gender + age, data) However, it looks like the linearity of the logit ...
0
votes
0answers
33 views

How does the convolution work for a simple example 1D and its relation to the true mathematical convolution?

I was trying to pin point precisely mathematically what the convolution does for a simple 1D example (i.e. $x \in \mathbb{R}^D$ as opposed to an image $x \in \mathbb{R}^{D_1 \times D_2}$). The ...
1
vote
0answers
391 views

What's the intuition behind variational learning in Deep NNs with attention mechanism

I'm trying to understand this paper: "Multiple Object Recognition With Visual Attention (Ba et al., 2015)", specifically I'm trying to understand section 3. which explains how the model is trained. ...
2
votes
0answers
78 views

Convolutional Neural Networks with Caffe and NEGATIVE IMAGES

When training a set of classes (let's say #clases (number of classes) = N) on Caffe Deep Learning (or any CNN framework) and I make a query to the caffemodel, I get a % of probability of that image ...
0
votes
0answers
24 views

How to design an objective function in Convolutional neural networks to classify unlabeled images?

Convolutional neural networks have been used in supervised learning such that it changes the weights $\Theta$ to minimize $(f(X;\Theta)-Y)^2$. However, for unlabeled data, how does one design a new ...
1
vote
1answer
32 views

Backtesting in neural network field

I'm new to the neural network field and I would like to understand how one can backtest a neural network trained with backpropagation methodology. Particularly, I have a time series dataset and I ...
1
vote
0answers
61 views

Difference between Time delayed neural networks and Recurrent neural networks

I would like to use a Neural Network to predict financial time series. I come from an IT background and have some knowledge of Neural Networks and I have been reading about these: TDNN RNN I have ...
0
votes
2answers
54 views

Are fixed bias neurons or biased neurons better?

When building an artificial neural network, there seems to be two differing philosophies in usage of biases. There are those groups that propose neural networks with a fixed bias neuron with a ...