Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network ...

learn more… | top users | synonyms

0
votes
0answers
7 views

TensorFlow: Implementing Spearman Distance as the Objective Function

In order to make my issue reproducible, I have generated the following .csv file using iris flower data set (10 arbitrary rows, all columns standard normalized) and ...
0
votes
0answers
13 views

LSTM forget gate weights go to infinity

I've implemented an LSTM using the standard equations and I've found that for some very simple data the LSTM thinks the best choice is to never forget anything (up to a certain point in the input ...
2
votes
1answer
9 views

Is there anything wrong with treating neural network bias as a node?

I've seen a lot of examples of neural networks where when they introduce the bias, they treat it as a node that always outputs 1, and then the nodes each have an individual weight for it, instead of ...
0
votes
1answer
28 views

What does mAP mean? [on hold]

I'm reading different papers about Neural Networks, which express results on certain datasets in a percentage value called mAP. Could anyone explain me the meaning of mAP?
0
votes
0answers
8 views

How to design RNN\VAE for sparse sequential data?

I am currently struggling with designing some Deep Learning algorithms that takes as an input sparse sequential data, since the system I try to model signals sparsely (once in a while). I try to ...
1
vote
0answers
21 views

When doing regression with a singled layered Neural Network, what activation function is the best one to choose?

I was training a singled layered (shallow) neural network as in: $$ f(x) = \sum^K_{k} c_k\theta(W_k x+b_k)$$ for regression (using squared error loss) or function approximation. I was wondering, is ...
0
votes
1answer
37 views

Why is “the true data generating process” similar to simulating the entire universe?

In the deep learning book by Bengio, Goofellow and Courville (http://www.deeplearningbook.org/) there is paragraph in the regularization chapter. "Deep learning algorithms are typically applied to ...
1
vote
0answers
9 views

RBF network normalization, standardization in MATLAB

First of all, is it good to do normalization and standardization for Radial basis neural network? Does MATLAB do that automatically for RBF? I've read that MATLAB does normalization and ...
2
votes
0answers
14 views

Does ReLU layer work well for a shallow network?

I am currently working on training a 5-layer neural network, and I got some problems with tanh layer and would like to try ReLU layer. But I found that it becomes even worse for ReLU layer. I am ...
1
vote
0answers
20 views

Adversarial learning gradient derivation

I'm working through Convolutional Neural Network paper here on adversarial learning and I'm having trouble with the derivative proof of adversarial logistic regression. The correct answer presented (...
0
votes
0answers
42 views
+50

Help in problem formuation :Hebb's learning

In the supervised learning problem, the goal is, given a training set, to learn a function $h : X \mapsto Y$ so that $h (x)$ is a “good” predictor for the corresponding value of $y$. If $y$ takes ...
1
vote
1answer
31 views

Should I optimize neural networks that are part of ensemble of neural networks?

I'm creating ensemble of neural networks for a simple binary classification task. Every neural network is generated and trained a bit differently (number of hidden layers, number of neurons per layer, ...
-1
votes
0answers
71 views

Prisma App Neural Networks [on hold]

Can anyone point me towards relevant material that would allow me to understand how Prisma works? http://prisma-ai.com/index.html
0
votes
0answers
14 views

Some pointers about training with large datasets

I'm trying to design a neural network (conv. neural networks, the so-called 'deep neural networks'). In this process I'm trying to think of a suitable design for my problem (in a natural language ...
2
votes
1answer
30 views

How do neural networks optimize functions?

This may be a stupid question, but I have been reading into Neural Networks with the aim of using the concepts to optimize a multi objective cost function (aka finding Pareto fronts) which takes a ...
0
votes
0answers
11 views

Recompute Neural Network output using weights from Matlab

I'm trying to recompute the output in excel given weights from Matlab's function of Neural Network. I'm working from the "House Pricing" example data set found in the "nftool" function. I selected 1 ...
0
votes
0answers
9 views

How common is it to train each network independently then train end-to-end in a neural network?

Say that I had 3 autoencoders stacked on each other. How common is it to ...
0
votes
0answers
13 views

Is it better to use noised (corrupted) data as validation set?

I ran some tests on Autoencoders and couldn't come to a conclusion, because to obtain a conclusive result hyperparameters should be tuned, as well. Instead, I looked for theorical justification of ...
2
votes
1answer
60 views

Why Neural Network is Failing in a simple classification case

I have the code below where a simple rule based classification data set is formed: ...
0
votes
0answers
10 views

How does depth-efficiency help neural networks learn?

Depth efficiency is an accepted result about neural networks that says the expressivity of a network with additional layers can only be matched by a shallow network with exponentially many more nodes. ...
2
votes
1answer
41 views

Do convolutional neural networks flip the kernel?

After reading various examples of CNNs it doesn't look like the kernel used for convolution is flipped. Can anybody explain why?
2
votes
0answers
25 views

Why doesn't the hidden state of a neuron network provide better dimension reduction result than original input?

I just read a great post here. I am curious about content of "An example with images" in that post. If the hidden states mean a lot of features of the original picture and getting closer to final ...
0
votes
0answers
23 views

Product suggestions based on what users have also bought

We need to train a model that when given an input of a product name and group, it outputs product suggestions for what users frequently buy together. We have product data stored like: ...
0
votes
0answers
7 views

How to create machine learning model for sparse 1D data?

I want to predict the frequency of Dataset 1 from the information in Dataset 2. Dataset 1: List of 1D points on a flat number line. Number line ranges from 1 to 3 billion (yes billion). Dataset 2: ...
2
votes
1answer
12 views

Best way to initialize LSTM state

I was wondering what is the best way to initialize the state for LSTMs. Currently I just initialize it to all zeros. I can not really find anything online about how to initialize it. One thing I ...
1
vote
0answers
15 views

time series classification of an event either happening or not happening using machine learning techniques

I have sensor data that I would like to use to classify whether an an event (giving birth) is about to occur within (2-4hrs) in an animal based on various metrics collected by the sensor(activity ...
0
votes
1answer
20 views

Neural network not i.i.d

Is it not considered correct to use non i.i.d samples in neural networks, for example if I have pixels in an image as observations? I have read that this is an assumption, but what if this is not the ...
0
votes
0answers
11 views

NN validation error decreases and becomes constant after a while. What does that mean?

I am using DNN with dropout(0.75), Adam, Relu. Is this overfitting or model gone haywire? Is using the model before it went haywire a good idea? Model is 4 hidden layer(1024), input size is 8192. ...
0
votes
0answers
21 views

Normality assumption neural networks

I am doing regression with neural network and my inputs and target data is not normal even after log transforming and standardizing. I am using MSE as the cost function. Is there a normality ...
0
votes
0answers
18 views

Why are LSTMs performing worse on the training set with higher cell size?

I'm attempting to train a (single-layer) LSTM-based autoencoder using the seq2seq framework within Tensor Flow. The input is sequences of numerical values. The sequences each contain 40 values. As a ...
1
vote
1answer
26 views

Error function for neural network, bimodal target data

I'm wondering if it's okay to use the error function mean squared error (MSE) when using the multilayer perceptron for a regression problem where the target data is bimodal. I standardize the inputs ...
0
votes
0answers
13 views

What is the type of measurment of cars' ages for scaling purposes?

I have a variable for the cars' ages which range from 1950 to 2016. I am not sure whether to treat the variable as ordinal (since I can order them)? interval (since I know the difference between every ...
4
votes
1answer
46 views

Does an optimally designed neural network contain zero “dead” ReLU neurons when trained?

In general should I retrain my neural network with fewer neurons so that it has fewer dead ReLU neurons? I've read conflicting opinions about dead ReLUs. Some sources say dead ReLUs are good because ...
3
votes
1answer
43 views

Neural network library for Python for Microsoft Windows

I have been having trouble with selecting a good library for Neural network algorithms in Python. TensorFlow isn't supported on windows. Theano is still in the beta phase of development. PyBrain too ...
0
votes
1answer
40 views

Vanishing gradient in basic 3-layer neural networks?

A 3-layer network has two layers of connections (between input and hidden layers and between hidden and output layer). Doesn't this mean that the gradient "vanishes", at least slighty, when training ...
1
vote
1answer
33 views

Mixing Convolutional Neural Networks and “regular” Feed Forward Neural Networks

I've just been watching some computerphile videos on ANNs. And in one of them the guy talked about figuring out the price of a house according to a picture. At that moment an idea came into my head. ...
0
votes
0answers
8 views

Derivation of back propagation though time

There is a wonderful explanation for the implementation of Backpropagation through time in the this article by Denny Britz here: http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-...
1
vote
1answer
20 views

Nesterov accelerated gradient descent in neural networks

I have a simple gradient descent algorithm implemented in MATLAB which uses a simple momentum term to help get out of local minima. ...
0
votes
2answers
60 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I stuck in some questions. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
2
votes
1answer
41 views

Different definitions of the cross entropy loss function

I started off learning about neural networks with the neuralnetworksanddeeplearning dot com tutorial. In particular in the 3rd chapter there is a section about the cross entropy function, and defines ...
0
votes
0answers
8 views

Approaching estimating a distribution

My task is given 101 prices (i.e for each day, so first price is for day one, second price for day two, etc), this curve needs to be judged on how much it fits 4 curves (these are preset). There is ...
0
votes
0answers
18 views

How to train a neural network without training data

I have a game which takes 4 inputs (4 directions), every move yields a score accordingly. As the game progresses the score increases. I want to train the neural network to take the inputs of the game ...
2
votes
3answers
58 views

Random initialization/order in neural network — bias or variance?

I'm puzzled about how to describe differences occurring between neural networks trained on the same data and with the same configuration. They differ only in the initial weights (different seeds used ...
0
votes
1answer
25 views

Decrease in filter size as CNN progresses?

I have noticed, as a trend, people seem to "taper" the size of their filters as a convolutional network progresses. By this I mean they begin convolving the image/patch with a larger filter, and ...
1
vote
0answers
31 views

A (simple) example where LSTM works but a regular Neural Network (NN) fails?

In the spirit of the answer from maple on this thread: Using RNN (LSTM) for predicting the timeseries vectors (Theano) I created some simple sine wave data to fit with a LSTM. It worked well! ...
2
votes
1answer
36 views

Dimensional mismatch of “sliding-windows” (convolutions, pooling etc)

As far as I can see, ResNet-152 (paper, visualization, Caffe Model) expects inputs with dimensions 224x224x3, and its first layer does 64 convolutions, each against a 7x7 kernel with a padding of 3 ...
1
vote
1answer
35 views

Error Backpropagation, Christopher Bishop “Pattern Recognition and Machine Learning”

I'm trying to understand the description of the error backpropagation algorithm as explained in Christopher Bishop's book, in particular, section 5.3.1 "Evaluation of error-function derivative". The ...
0
votes
0answers
8 views

implementing contractive autoencoder weight decay

I'm trying to implement the idea behind the contractive auto-encoder and penalty the jacobian of output of an MLP wrt its input to get make it less sensitive to noise. there is a scalar coefficient ...
0
votes
0answers
25 views

How to calculate error for backpropagation for neural networks with multiple hidden layers

I am trying to make my own neural network. I wanted it to work with multiple hidden layers. I'm having trouble and I don't get why it isn't working. I am calculating the error by subtracting the ...
1
vote
1answer
84 views

Convergence Criteria for Stochastic Gradient Descent

I am familiar with the update rule in SGD whereby the weights ($\theta$) are updated with the gradients of the cost function ($\nabla Q$) for each sample times the learningrate ($\eta$). $$ w := w - \...