Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network ...

learn more… | top users | synonyms

0
votes
0answers
6 views

R darch : deep neural net pre-training

I'm trying to train a (not even deep) network on iris using the darch package. It works well as a standard backprop-net, but becomes bad as soon as I add a little pre-training: ...
1
vote
1answer
17 views

Can you determine how a neural network produces its results?

I read an article about neural networks that stated you really can't determine how the network produced a given result. What I mean is that there might be several thousands of factors behind the ...
0
votes
1answer
9 views

Cross entropy loss function and division by zero

I'm trying out the cross entropy loss function for neural network training, per the arguments at ...
1
vote
0answers
20 views

What is the minimum sample size required to train a Deep Learning model - CNN?

It is true that the sample size depends on the nature of the problem and the architecture implemented. But, on average, what is the typical sample size utilized for training a deep learning framework? ...
0
votes
0answers
10 views

canonical benchmark on convergence measures for neural networks

This is in reference to an answer to a previous question (here), and to a related question (here). I know there are a truck-load of data sets out there (link). I know there is very wide variety of ...
0
votes
0answers
9 views

Denoising Autoencoder not training properly

I've implemented a denoising autoencoder using TensorFlow. The code is here, there is also a command line script to launch it. The code seems to work, the cross-validation error is decreasing every ...
0
votes
1answer
25 views

SLP vs. MLP: Is my data linearly separable?

I implemented an artificial neural network using scikit neuralnetwork. As default configuration for my classification task I am using 10730 Datsets x 115 Features 1 Hidden Layer with 61 neurons 7 ...
0
votes
1answer
39 views

Neural network: two output vectors?

Architecture: I have a CNN which does some classification for me. The output layer y consists of a vector $\vec{y}$ which is of dimension $(1, 1000)$, so it has 1.000 neurons in total (the weight ...
3
votes
1answer
22 views

Why does SGD and back propagation work with ReLUs?

ReLUs are not differentiable at the origin. However, they are widely used in Deep Learning together with Stochastic Gradient Descent algorithms and Backpropagation, where the gradients of the loss ...
0
votes
0answers
8 views

Identifying important differences between supervised learning datasets

The training data in a multi-class supervised learning task shows a significant dependence on time that is apparently not captured well by my learners. Specifically, the two learners I used (OvR ...
3
votes
0answers
25 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
1
vote
1answer
30 views

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what ...
-5
votes
0answers
23 views

Artificial neural network [on hold]

i'm interested in modeling crop response employing neural network but i don't know how to feed new/fresh sets of data after training. can anybody enlighten me with this? thanks.
0
votes
0answers
14 views

Unknown Variable in Neural Network

I was reading a paper from 1996 (http://www2.cs.uregina.ca/~jtyao/Papers/marketing_jisi.pdf) where an Unknown variable was used in the ANN that apparently caught information and influences not ...
0
votes
1answer
16 views

How to find the variance in this neural network related question?

I have been going through Neural Networks and Deep Learning. There is a way to represent the activation of network as: z = summation of(w*x) + b where w,b are weight and bias with mean of 0 and S.D ...
1
vote
0answers
6 views

Deriving gradient of a single layer neural network w.r.t its inputs, what is the operator in the chain rule?

Problem is: Derive the gradient with respect to the input layer for a a single hidden layer neural network using sigmoid for input -> hidden, softmax for hidden -> output, with a cross ...
1
vote
1answer
27 views

Resources to get started with deep reinforcement learning

Assume the learner is proficient with artificial neural networks, and has some background in reinforcement learning. What are some good resources (books/videos/papers/GitHub repo/etc.) to get started ...
0
votes
0answers
12 views

Looking for a CNN implementation for 3D images

I'm looking for an implementation in python (or eventually matlab) of Convolutional Neural Networks for 3D images. By 3D I mean 3 spatial dimensions (i.e. not 2D+channels or 2D+time). Any advice?
0
votes
2answers
30 views

Classification with a neural network when one class has disproportionately many entries

I try to train a neural network using a dataset with several classes $c_1, c_2, \dotsc, c_{10}$. The class $c_1$ has a lot more entries in the training set than the other classes, and this makes my ...
2
votes
1answer
30 views

Neural network vectorization, no convergence

2I am trying to implement a fully vectorized neural net following this example Stanford. I am using C# with Math.net numerical library of which the binaries can be downloaded here NuGet. Because the ...
1
vote
1answer
25 views

Neural network - continuous vs. non continuous variables

I'm using the neuralnet package in R to attempt to predict the median value of Sales using all the other variables of the data ...
0
votes
0answers
4 views

how to intrepret the results from a diebold-mariano test statistic [closed]

my goal is to interpret the findings after running the test statistic in Mat lab. when I ran the test statistic I got a figure 0.853. what does that figure stand for?
0
votes
0answers
16 views

Suggestions for Neural Network Structure for Time-Series prediction with constant covariates

I've been working on a time series prediction problem and wondered if someone has run across a similar problem structure & can make a suggestion on how to structure the training data, network, or ...
0
votes
0answers
12 views

Neural network, for anomalous signature matching

I have a large dimensional dataset. It can be safely assumed that 99.99...% of the dataset contains mostly uninteresting background data by definition. What I care about is designing an algorithm ...
0
votes
0answers
18 views

Evaluating performance Neural Network embeddings in kNN classifier

I am solving a classification problem. I train my unsupervised neural network for a set of entities (using skip-gram architecture). The way I evaluate is to search k nearest neighbours for each point ...
0
votes
0answers
26 views

Struggling to train a MLP using Keras (Python)

I've been interested in NNs for a while, just started playing with them. I liked the look of Keras, so I got started with some toycode to do some regression. I tried the simplest set up I could: ...
5
votes
0answers
32 views

State of the art in general learning from data in '69

I'm trying to understand the context of the famous Minsky and Papert book "Perceptrons" from 1969, so critical to neural networks. As far as I know, there were no other generic supervised learning ...
0
votes
0answers
18 views

Application fields of Neural Network and backpropagation algorithm - frontiers of Artificial Intelligente

I am studying some neural networks and, as far as I understood, the backpropagation algorithm is one of the fundamental learning algorithm. Up to now I have encountered that this algorith is applied ...
0
votes
0answers
17 views

Neural network + probability distribution modeling

I'm having A training set $(a^i_1, a^i_2...a^i_n, t)$ I also have a probability distribution over $a_k$ and $t$ I need to use a neural network in order to find an aproximation. How can I include ...
2
votes
0answers
34 views

In RNN Back Propagation through time, why is the D(h_t)/D(h_(t-1)) diagonal?

I was going through backpropagation in time for RNN, in the deep learning book of Joshua Bengio et.al. (deep learning book , section 10.2.1 ). Given a network as: the book tells that the ...
1
vote
3answers
35 views

Backpropagation: Is there a general weight update rule for both output and hidden layers?

I'm looking for a general weight update rule for both hidden and output layers, no matter the number of layers, the connections or the transfer function. Does anything like this exist? I'm quite new ...
0
votes
0answers
23 views

How to train neural net to output a 2D polygon?

Lets consider a net, that should map image to an arbitrary four-sided 2D polygon (all vertices are scaled to (0,0) - (1,1)). We need a loss function with a gradient to train it. If we try to use some ...
2
votes
1answer
42 views

What is pre training a neural netwok

Well the question says it all. What is meant by pre training a neural network. Can someone explain in pure simple english. I can't seem to find any resources related to it. It would be great if ...
0
votes
1answer
24 views

How to train and fine-tune fully unsupervised deep neural networks?

In scenario 1, I had a multi-layer sparse autoencoder that tries to reproduce my input, so all my layers are trained together with random-initiated weights. Without a supervised layer, on my data this ...
2
votes
0answers
23 views

How do I calculate the derivative of kurtosis and entropy?

I'm using kurtosis and entropy as penalty terms in my neural network's cost function. Need to back-propagate the error. For that I need a quick way to estimate the derivative (the gradient ...
2
votes
2answers
33 views

Encoding Date/Time (cyclic data) for Neural Networks

First of all I'm very impressed by this community and some of the answers here. That makes me a bit shy to ask such a simple sounding question: How to encode the date and time of an event for a ...
0
votes
0answers
42 views

Multi label image classification using convolutional neural network in Python

I am working on multi label image classification problem. The dataset is given on this link. I am using Convolutional Neural Network (CNN) with fully connected neural network (NN) at the end. I am ...
0
votes
0answers
28 views

Online Machine Learning of sequential events with varying delay

Lets say we have A to Z features which repeat sequentially. So you have A(1), B(1), ... Z(1) at time 1 followed by A(2), B(2),....Z(2) at time 2 and so on till A(n), B(n), ... Z(n) at time n. Each of ...
0
votes
0answers
19 views

What will be the training dataset for learning a map using neural network

I am new to neural networks and training and finding it hard to understand how I can train the Neural Network (NN) in learning a time series generated by a non-linear discrete map $f : I \rightarrow I ...
1
vote
0answers
20 views

Using an RNN/LSTM to generate sequences with a unique output

I'm trying to train a LSTM recurrent neural network where my data consists of a sequence of animal migration data ...
1
vote
1answer
33 views

neural networks - Inputting a time series to a classification NN

I have a simple ANN that does the job of classification between two labels-: Sick Healthy What I want to do is that input patient data ie. heart rate(ECG), EEG, etc which will be in the form of a ...
2
votes
1answer
23 views

When implementing dropout in neural networks with SGD, how does one calculate the gradient?

Specifically, I know that in SGD one sums all the gradients for weights/biases for each minibatch and divides by the mini batch size, would one do the same thing for dropout networks? Or would they ...
1
vote
1answer
30 views

What prevents duplication of neuron parameters in fully connected layer

LeNet has several fully connected layers, I'm wondering what prevents neurons duplicate other's weights and outputs. Unfortunately the only technique, I came up with, is random weight initialization ...
1
vote
1answer
38 views

CNN filter sizes and padding

I came across the following passage in http://cs231n.github.io/convolutional-networks/ with regards to filter sizes in CNNs. Purely because i have seen a number of networks with 5*5 conv filters ...
0
votes
1answer
63 views

Magic of neural networks?

I've read a bit on neural networks and wonder why they are actually so popular at the moment. For example, a feed-forward network is a quite simple mathematical object: An input vector $\mathbf{x}\in ...
1
vote
1answer
40 views

Neural networks with complex weights

I am currently wishing to give a neural network starting weights with complex values (because of the nature of the specific task I am working with). I was trying to use the standard neural net ...
0
votes
1answer
47 views

SVM Vs Neural Network Vs Random Forest classifier comparison on multi class problem

Any idea if SVM or Neural Net or Random Forest works better on a classification problem on the same multi class dataset? I mean, in general, which should outperform the comparison?
0
votes
1answer
59 views

Deep neural network - How many layers?

I am trying to implement a multi-layer deep neural network (over 100 layers) for image recognition. As far as i can understand each layer learns specific features. I am feeding 100x100 pixel color ...
0
votes
0answers
11 views

Correct output labelling for RNNs/ANNs with sequential data and single label

I have a labelled time-series dataset; for example, a sequential input length $n$ from $k$ sensors corresponds to class '1' (see below for $n=50, k=2$). How can I correctly classify it? If I use RNN ...
0
votes
1answer
38 views

neural networks - What is meant by “linear combination of inputs”

Just starting out with MLPs. I am reading a tutorial that I found here. It says that the disadvantages of using a linear function is that the neural net will only be restricted to learning "linear ...