# Tagged Questions

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network ...

7 views

### TensorFlow: Implementing Spearman Distance as the Objective Function

In order to make my issue reproducible, I have generated the following .csv file using iris flower data set (10 arbitrary rows, all columns standard normalized) and ...
13 views

### LSTM forget gate weights go to infinity

I've implemented an LSTM using the standard equations and I've found that for some very simple data the LSTM thinks the best choice is to never forget anything (up to a certain point in the input ...
9 views

### Is there anything wrong with treating neural network bias as a node?

I've seen a lot of examples of neural networks where when they introduce the bias, they treat it as a node that always outputs 1, and then the nodes each have an individual weight for it, instead of ...
28 views

### What does mAP mean? [on hold]

I'm reading different papers about Neural Networks, which express results on certain datasets in a percentage value called mAP. Could anyone explain me the meaning of mAP?
8 views

### How to design RNN\VAE for sparse sequential data?

I am currently struggling with designing some Deep Learning algorithms that takes as an input sparse sequential data, since the system I try to model signals sparsely (once in a while). I try to ...
21 views

### When doing regression with a singled layered Neural Network, what activation function is the best one to choose?

I was training a singled layered (shallow) neural network as in: $$f(x) = \sum^K_{k} c_k\theta(W_k x+b_k)$$ for regression (using squared error loss) or function approximation. I was wondering, is ...
37 views

### Why is “the true data generating process” similar to simulating the entire universe?

In the deep learning book by Bengio, Goofellow and Courville (http://www.deeplearningbook.org/) there is paragraph in the regularization chapter. "Deep learning algorithms are typically applied to ...
9 views

### RBF network normalization, standardization in MATLAB

First of all, is it good to do normalization and standardization for Radial basis neural network? Does MATLAB do that automatically for RBF? I've read that MATLAB does normalization and ...
14 views

### Does ReLU layer work well for a shallow network?

I am currently working on training a 5-layer neural network, and I got some problems with tanh layer and would like to try ReLU layer. But I found that it becomes even worse for ReLU layer. I am ...
20 views

I'm working through Convolutional Neural Network paper here on adversarial learning and I'm having trouble with the derivative proof of adversarial logistic regression. The correct answer presented (...
42 views
+50

### Help in problem formuation :Hebb's learning

In the supervised learning problem, the goal is, given a training set, to learn a function $h : X \mapsto Y$ so that $h (x)$ is a “good” predictor for the corresponding value of $y$. If $y$ takes ...
31 views

### Should I optimize neural networks that are part of ensemble of neural networks?

I'm creating ensemble of neural networks for a simple binary classification task. Every neural network is generated and trained a bit differently (number of hidden layers, number of neurons per layer, ...
71 views

### Prisma App Neural Networks [on hold]

Can anyone point me towards relevant material that would allow me to understand how Prisma works? http://prisma-ai.com/index.html
14 views

### Some pointers about training with large datasets

I'm trying to design a neural network (conv. neural networks, the so-called 'deep neural networks'). In this process I'm trying to think of a suitable design for my problem (in a natural language ...
30 views

### How do neural networks optimize functions?

This may be a stupid question, but I have been reading into Neural Networks with the aim of using the concepts to optimize a multi objective cost function (aka finding Pareto fronts) which takes a ...
11 views

### Recompute Neural Network output using weights from Matlab

I'm trying to recompute the output in excel given weights from Matlab's function of Neural Network. I'm working from the "House Pricing" example data set found in the "nftool" function. I selected 1 ...
9 views

### How common is it to train each network independently then train end-to-end in a neural network?

Say that I had 3 autoencoders stacked on each other. How common is it to ...
13 views

### Is it better to use noised (corrupted) data as validation set?

I ran some tests on Autoencoders and couldn't come to a conclusion, because to obtain a conclusive result hyperparameters should be tuned, as well. Instead, I looked for theorical justification of ...
60 views

### Why Neural Network is Failing in a simple classification case

I have the code below where a simple rule based classification data set is formed: ...
10 views

### How does depth-efficiency help neural networks learn?

Depth efficiency is an accepted result about neural networks that says the expressivity of a network with additional layers can only be matched by a shallow network with exponentially many more nodes. ...
41 views

### Do convolutional neural networks flip the kernel?

After reading various examples of CNNs it doesn't look like the kernel used for convolution is flipped. Can anybody explain why?
25 views

### Why doesn't the hidden state of a neuron network provide better dimension reduction result than original input?

I just read a great post here. I am curious about content of "An example with images" in that post. If the hidden states mean a lot of features of the original picture and getting closer to final ...
23 views

### Product suggestions based on what users have also bought

We need to train a model that when given an input of a product name and group, it outputs product suggestions for what users frequently buy together. We have product data stored like: ...
7 views

### How to create machine learning model for sparse 1D data?

I want to predict the frequency of Dataset 1 from the information in Dataset 2. Dataset 1: List of 1D points on a flat number line. Number line ranges from 1 to 3 billion (yes billion). Dataset 2: ...
12 views

### Best way to initialize LSTM state

I was wondering what is the best way to initialize the state for LSTMs. Currently I just initialize it to all zeros. I can not really find anything online about how to initialize it. One thing I ...
15 views

### time series classification of an event either happening or not happening using machine learning techniques

I have sensor data that I would like to use to classify whether an an event (giving birth) is about to occur within (2-4hrs) in an animal based on various metrics collected by the sensor(activity ...
20 views

### Neural network not i.i.d

Is it not considered correct to use non i.i.d samples in neural networks, for example if I have pixels in an image as observations? I have read that this is an assumption, but what if this is not the ...
11 views

### NN validation error decreases and becomes constant after a while. What does that mean?

I am using DNN with dropout(0.75), Adam, Relu. Is this overfitting or model gone haywire? Is using the model before it went haywire a good idea? Model is 4 hidden layer(1024), input size is 8192. ...
21 views

### Normality assumption neural networks

I am doing regression with neural network and my inputs and target data is not normal even after log transforming and standardizing. I am using MSE as the cost function. Is there a normality ...
18 views

### Why are LSTMs performing worse on the training set with higher cell size?

I'm attempting to train a (single-layer) LSTM-based autoencoder using the seq2seq framework within Tensor Flow. The input is sequences of numerical values. The sequences each contain 40 values. As a ...
26 views

### Error function for neural network, bimodal target data

I'm wondering if it's okay to use the error function mean squared error (MSE) when using the multilayer perceptron for a regression problem where the target data is bimodal. I standardize the inputs ...
13 views

### What is the type of measurment of cars' ages for scaling purposes?

I have a variable for the cars' ages which range from 1950 to 2016. I am not sure whether to treat the variable as ordinal (since I can order them)? interval (since I know the difference between every ...
46 views

### Does an optimally designed neural network contain zero “dead” ReLU neurons when trained?

In general should I retrain my neural network with fewer neurons so that it has fewer dead ReLU neurons? I've read conflicting opinions about dead ReLUs. Some sources say dead ReLUs are good because ...
43 views

### Neural network library for Python for Microsoft Windows

I have been having trouble with selecting a good library for Neural network algorithms in Python. TensorFlow isn't supported on windows. Theano is still in the beta phase of development. PyBrain too ...
40 views

### Vanishing gradient in basic 3-layer neural networks?

A 3-layer network has two layers of connections (between input and hidden layers and between hidden and output layer). Doesn't this mean that the gradient "vanishes", at least slighty, when training ...
33 views

### Mixing Convolutional Neural Networks and “regular” Feed Forward Neural Networks

I've just been watching some computerphile videos on ANNs. And in one of them the guy talked about figuring out the price of a house according to a picture. At that moment an idea came into my head. ...
8 views

### Derivation of back propagation though time

There is a wonderful explanation for the implementation of Backpropagation through time in the this article by Denny Britz here: http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-...
20 views

### Nesterov accelerated gradient descent in neural networks

I have a simple gradient descent algorithm implemented in MATLAB which uses a simple momentum term to help get out of local minima. ...
60 views

### how to propagate error from convolutional layer to previous layer？

I've been trying to implement a simple convolutional neural network. But I stuck in some questions. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
41 views

### Different definitions of the cross entropy loss function

I started off learning about neural networks with the neuralnetworksanddeeplearning dot com tutorial. In particular in the 3rd chapter there is a section about the cross entropy function, and defines ...
8 views

### Approaching estimating a distribution

My task is given 101 prices (i.e for each day, so first price is for day one, second price for day two, etc), this curve needs to be judged on how much it fits 4 curves (these are preset). There is ...
18 views

### How to train a neural network without training data

I have a game which takes 4 inputs (4 directions), every move yields a score accordingly. As the game progresses the score increases. I want to train the neural network to take the inputs of the game ...
58 views

### Random initialization/order in neural network — bias or variance?

I'm puzzled about how to describe differences occurring between neural networks trained on the same data and with the same configuration. They differ only in the initial weights (different seeds used ...
25 views

### Decrease in filter size as CNN progresses?

I have noticed, as a trend, people seem to "taper" the size of their filters as a convolutional network progresses. By this I mean they begin convolving the image/patch with a larger filter, and ...
31 views

### A (simple) example where LSTM works but a regular Neural Network (NN) fails?

In the spirit of the answer from maple on this thread: Using RNN (LSTM) for predicting the timeseries vectors (Theano) I created some simple sine wave data to fit with a LSTM. It worked well! ...
36 views

### Dimensional mismatch of “sliding-windows” (convolutions, pooling etc)

As far as I can see, ResNet-152 (paper, visualization, Caffe Model) expects inputs with dimensions 224x224x3, and its first layer does 64 convolutions, each against a 7x7 kernel with a padding of 3 ...
35 views

### Error Backpropagation, Christopher Bishop “Pattern Recognition and Machine Learning”

I'm trying to understand the description of the error backpropagation algorithm as explained in Christopher Bishop's book, in particular, section 5.3.1 "Evaluation of error-function derivative". The ...
8 views

### implementing contractive autoencoder weight decay

I'm trying to implement the idea behind the contractive auto-encoder and penalty the jacobian of output of an MLP wrt its input to get make it less sensitive to noise. there is a scalar coefficient ...
I am familiar with the update rule in SGD whereby the weights ($\theta$) are updated with the gradients of the cost function ($\nabla Q$) for each sample times the learningrate ($\eta$).  w := w - \...