Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network ...

learn more… | top users | synonyms

0
votes
0answers
5 views

should i be treating variable for non-linearity for neural network model?

We usually treat the variables for regression analysis for any non-linearity. Now since we fit a non-linear function in NNet, should we be treating the variables for linearity before feeding into ...
0
votes
0answers
14 views

Cost functions like cross-entropy in backpropagation for non sigmoid activation?

I was following this resource. Cross-entropy function was introduced as cost function. When calculating gradient in backpropagation we get delta values which depend on derivation of activation ...
0
votes
0answers
5 views

spss neural network forecasting with lags

I have 240 monthly data points and would like to leave 36 out of sample for neural network forecasting in SPSS. I made the covariates as lag of 1 [AR(1)] and have several questions to ask: What are ...
2
votes
1answer
19 views

regarding the convolutional network structure of FCNN

The paper of Fully Convolutional Networks for Semantic Segmentation , gives the following image, . What do those numbers represent, 96, 256, 384, etc? Are them ...
3
votes
1answer
23 views

Distribution of classes in neural network batches

I'm creating a neural network for classifying input data. When using batches, do I need to ensure a somewhat uniform distribution of classes per batch? Or can I simply split up my data in any which ...
0
votes
0answers
19 views

Difference between particle filter (PF) and recurrent neural network (RNN) for time series

Both method are used to estimate time series from data. The question is, when should I use one method or other? Is any advantage to use one instead of the other? I know that in a PF there is a hidden ...
1
vote
1answer
20 views

How to deal with discrete and continuous output multi variables in neural network?

I have created neural networks using nnet for either discreate or continous output variables, but not using both at once. Now I have a problem in which the output ...
6
votes
0answers
43 views

Restricted Boltzmann Machine : how is it used in machine learning?

Background: Yes, Restricted Boltzmann Machine (RBM) CAN be used to initiate the weights of a neural network. Also it CAN be used in a "layer-by-layer" way to build a deep belief network (that is, to ...
0
votes
0answers
10 views

Is using pairwise comparison instead of single label value a valid way to augment training data

There are many papers using siamese architecture to do pairwise ranking. For example: suppose my training set is $\{X,Y\}$, where $Y$ is the continuous value. I can find a regressor to minimize $\|h(X)...
1
vote
1answer
29 views

Learning just a decoder (autoencoder without encoder)

I am trying to do something quite unusual: learning a latent representation of some data just by optimizing a decoder. Basically, a probabilistic model of a neural network autoencoder without the ...
1
vote
0answers
16 views

Increasing the learning rate on loss function saturation

I'm currently reading about neural networks, specifically how loss functions saturation can cause problems. During my studies, I was curious if one could remedy the problem during training of neural ...
1
vote
0answers
33 views

What is the FASTEST Neural Network Command in R? [on hold]

I'm running Neural Network on a data frame with 40,000 observations, 7500 predictors and with one response variables. The response variable is a categorical variable with 4 levels. I've found a lot ...
0
votes
0answers
37 views

convert local to saddle in Deep neuralnetwork

I am newbie in deep NN. Hope you can suggest me a hint. Is it possible to transform local minimum points to saddle points in deep neural network? How can i do it?
3
votes
2answers
52 views

Can I use drop connect with sigmoid activation

The literature discusses tanh and relu activation. Does drop connect not work with sigmoid activation?
0
votes
1answer
27 views

How are hidden layer weights computed in a multilayer neural network?

More specifically, given a typical neural network with a single hidden layer $Z_m$ where $m = 1,...,M$ (see specifications/notation below drawn from p. 392 of Elements of Statistical Learning (Hastie, ...
2
votes
1answer
19 views

What will be the input value (i.e. $x$ and $x′$) of RBF kernel for a given dataset or data matrix $x$?

If $x$ is a data matrix or dataset then What will be the input value (i.e. $x$ and $x'$) of RBF kernel $K_r(x,x')=\exp(-\frac{\|x-x'\|^2}{r})$ ? I can understand $x$ is same as dataset or data matrix ...
1
vote
0answers
27 views

Training a neural network on one answer only

Let's say, I have a neural network and I'd like to learn it to determine weather a player (in any first person shooter of choice) is using an aim bot or not. I started thinking of a way to make ...
0
votes
0answers
19 views

Standardizing skewed boolean neural net inputs

I am developing an artificial neural network with a large number of inputs. Some of the boolean inputs are highly skewed: they have positive values less than 1 in 10000 or even worse. I am ...
3
votes
0answers
26 views

Q-learning with Neural Networks: one output unit per action

I am using Neural Network Q-value approximation in my reinforcement learning task. The approach is exactly the same as one described in this question, however the question itself is different. In ...
1
vote
0answers
21 views

What is the effect of number of hidden layers on local optimas in neural networks?

How does the number of hidden layers, affect local minimas? is it because of only the number of parameters or the depth of the network? for example a network of 5 hidden layer with each having 500 ...
1
vote
1answer
10 views

Rescaling weights after drop connect

From what I understand, you're supposed to rescale your activation layer after an application of dropout by an amount proportional to how much you dropped. Essentially truing up the lost relevance (...
1
vote
1answer
12 views

Drop Connect Back Propagation

I'm trying to implement drop connect. Am I supposed to use the same drop masks during back propagation?
0
votes
0answers
7 views

How do I perform backpropagation in a Convolutional Neural Network with 2 convolutional layers?

Specifically, I'm using a CNN for image classification and its architecture is: data->conv1->pool1->conv2->pool2->classifier/softmax When propagating errors from conv2 to pool1, how do I do that? I ...
3
votes
1answer
67 views

Should I use drop connect and L2 regularization?

I've just learned about the drop connect technique for neural networks. It is my understanding that it is intended to reduce over fitting and is referred to as regularization. Is there additional ...
0
votes
0answers
15 views

Unable to recreate the sine function in Figure 5.3 - Pattern recognition and machine learning (Bishop)

I'm trying to recreate the sine function according to Figure 5.3. Of course since in the range [-1;1] this function won't look like the one in the figure (because the closest minima and maxima are x = ...
1
vote
0answers
16 views

Pattern recognition and machine learning (Bishop) - Figure 5.3: Something is wrong with the sine function

In Figure 5.3, Pattern recognition and machine learning (Bishop), the author says he fitted 4 function: f(x) = x^2; f(x) = sin(x) ; f(x) = abs(x); f(x) = Heaviside(x), using 50 points chosen uniformly ...
1
vote
0answers
26 views

Understanding the weights of a neural network [duplicate]

I've been working with an artificial neural network in Torch, to solve a binary classification problem. I've had several troubles by using the mini-batch gradient update. Each element of my dataset ...
0
votes
1answer
41 views

How to use machine learning to determine if user signing in from a random location is an anomaly?

We have data that shows usernames and their IP addresses when they connect to a particular server. The data also contains IP address to geolocation mappings. So our data also contains fields that show ...
4
votes
1answer
51 views

Adam: stochastic gradient descent?

I would like to get a better idea of stochastic gradient descent algorithms, especially and most important Adam, since I've expierenced reasonable results with Adam and refuse to use something "just ...
2
votes
0answers
32 views
+50

How does one interpret histograms given by TensorFlow in TensorBoard?

I recently was running and learning tensor flow and got a few histograms that I did not know how to interpret. Usually I think of the height of the bars as the frequency (or relative frequency/counts)....
1
vote
0answers
20 views

Sample dependency in Neural Net Training cross-validation

I've created a Monte Carlo simulation that randomly divides my data into "test" and "training"-Samples and then trains a neural network. The ratio of 0 and 1 (19.62%) Category is stabilized on ...
0
votes
0answers
9 views

Tuning hyperparameters in light of a very large dataset

I have a rather large dataset (1M training samples). Each epoch in my neural network takes about 12 hours. I'm wondering what is the best strategy to tune the hyperparameters (batch size, step size, ...
3
votes
0answers
33 views

Should trading result be the same for data in reverse order in simple NN?

I have a simple neural net that takes an input (20 day price change) and tries to predict the future price change of a stock (A). On top of this is a simple rule to go either long or short. Just to ...
0
votes
2answers
20 views

Does backpropagation of error in multilayer neural network depend on the objective function being optimized?

I know for square error gradient descent is equivalent to back propagation. However, for a general objective function, how can I convince myself that back propagation is equivalent to computing ...
1
vote
1answer
47 views

RNN learning sine waves of different frequencies

As a warm up with recurrent neural networks, I'm trying to predict a sine wave from another sine wave of another frequency. My model is a simple RNN, its forward pass can be expressed as follow: $$ \...
0
votes
0answers
11 views

Torch - SparseLinear nn to handle large inputs and large output for a prediction problem

I'm pretty new to the magic of torch7 and seek your help/advice for a problem of mine. Context: I am working on a prediction problem. We observed a certain pattern in our values and would like to ...
0
votes
1answer
25 views

Input layer in Neural Network with different vocabulary size

I want to build a Neural Network where each neuron in the input layer will have different size. For instance, x1 = [0 1 0], x2 = [13 4 5 1 9 0], x3 = 7... Would it be possible to train this model? ...
0
votes
1answer
41 views

Why does Geoffrey Hinton say in his Coursera course that gradient magnitudes can vary widley when training Neural Networks?

I was watching his coursera course video on RMSProp and he said in a paraphrase: Gradients magnitude vary widely. I was wondering, why is it that they vary widely? I had a guess but wanted to ...
3
votes
0answers
44 views

Perceptron trained on time series always predicting the same answer

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
0
votes
0answers
11 views

Doc2Vec giving very less accuracy - 0.0002 [closed]

Using this tutorial - http://sujitpal.blogspot.in/2016/04/predicting-movie-tags-from-plots-using.html I am using the same code for a slightly different problem such that each doc has exactly one tag ...
1
vote
0answers
30 views

How does batch normalization compute the population statistics after training?

I was reading the batch normalization (BN) paper (1) and it said: For this, once the network has been trained, we use the normalization $$\hat{x} = \frac{x - E[x]}{ \sqrt{Var[x] + \epsilon}}$$ ...
1
vote
1answer
49 views

How and why does Batch Normalization use moving averages to track the accuracy of the model as it trains?

I was reading the batch normalization (BN) paper (1) and didn't understand the need to use moving averages to track the accuracy of the model and even if I accepted that it was the right thing to do, ...
0
votes
0answers
52 views

Free space detection on image

I have set of images with front or back rear cars which was obtained by cascade detector. I need to detect free space on image, (more precisely I need just to know how big car is in pixels). ...
2
votes
1answer
68 views

Analogy between Neural network and naive bayes

I am trying to understand the analogy between a single layer neural network and naive Bayes classifier. Particularly, I want to know if, in a neural network, the variables are independent given the ...
0
votes
0answers
18 views

removing outliers in poisson distributed data (count data) to improve classifier accuracy [on hold]

I wanted to know how to remove outliers in count data, the data i have is labeled can I remove the outliers from each class ? Or consider only 80% of the data close to the cluster center to train the ...
0
votes
0answers
12 views

How to set up neural network for finding text area?

I use neuroph and I want to find a text in an image. It is not even a photograph text but a computer painted java window with the same font except varying sizes/bolds or italics. My image is 300x150, ...
1
vote
0answers
11 views

How varying batch size in NN exactly influences the training results?

I was wondering how (if at all) a smaller batch size will influence the quality of the learning. What exactly happens in the NN when you decrease the batch size, except that it's more cost efficient?
0
votes
1answer
24 views

Perfect recall and spurious states in hopfield networks

In Hopfield networks, one can apparently load perfect recall into the network (by having enough neurons compared to patterns). (Source) However, at the same time, it appears that spurious states (i.e....
0
votes
1answer
39 views

Is it possible that a single neural network structure can solve multiple problems?

I am an enthusiast of neural network and regularly I try to model simple models in NN. While thinking about neural network's application in artificial intelligence I had this doubt arise in my mind ...
2
votes
0answers
20 views

What preprocessing techniques work well for autoencoding audio?

I am wondering what preprocessing techniques work well for autoencoding audio data? Specifically I have a dataset of ~0.5 second audio samples of people pronouncing digits 0-9 (think an audio version ...