Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
0answers
9 views

N-gram learning vs stochastic learning

I'm interested in comparing the differences in learning in n-grams and gradient-based learning (in my case with neural networks), particularly in the context of language modelling with the two classes ...
0
votes
1answer
33 views

Neural network & Bayesian in this machine learning algorithm

I am new to machine learning etc and found this comprehensive algorithm: http://scikit-learn.org/stable/tutorial/machine_learning_map/ . However, I am not able to make out any reference to neural ...
2
votes
0answers
44 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
1
vote
0answers
9 views

Is there any procedure to determine the number of layers of convolution and pooling needs in CNN?

When I want to use caffe lib to create my own CNN, how to determine the number of convolution and pooling layer I need is suitable to extract correct features basis? Is there any principal or ...
1
vote
2answers
66 views

Why convolutional neural networks belong to deep learning?

In my idea, deep learning is a process of feature extraction. Just like multiple layer neural networks (NN): ...
0
votes
0answers
16 views

Dynamic Neural Network with randomly shuffled training data

I'm using an Nonlinear AutoRegressive neural network with eXternal input (NARX) to model a water conveyance system as part of my master thesis. I'm implementing this model with MATLAB R2013b. In ...
2
votes
2answers
53 views

Using Neural Net weights as input to another classifer

Is there anyway to use the weights from a neural net hidden layer as input to another classifier, say a random forest? Of course this is trivial for the training data but how to score new data? Are ...
1
vote
0answers
10 views

Number of valid weight combinations in an ANN

Is it correct to assume that there is an infinite number of combinations of weights that a neural network can have in its connections in order to produce a specific output when given a certain set of ...
0
votes
0answers
16 views

What are the benefits of using ReLU over softplus as activation functions?

It is often mentioned that rectified linear units (ReLU) have superseded softplus units because they are linear and faster to compute. Does softplus it still have the advantage of inducing sparsity or ...
0
votes
0answers
24 views

How do you determine neural network loss function when there are multiple outputs?

This great Youtube tutorial taught me how to fit a neural network with one output. To apply back-propagation, you first find the Jacobian of the loss function with respect to the weights. ...
2
votes
0answers
16 views

Neural Network Learning Curves with Low Test Set Error

TLDR: You can see my neural network learning curves here: http://imgur.com/0CL6LVY. Which regularization term would you pick given that the test error actually drops below the training error at some ...
6
votes
2answers
218 views

Can we use MLE to estimate Neural Network weights?

I just started to study about stats and models stuff. Currently, my understanding is that we use MLE to estimate the best parameter(s) for a model. However, when I try to understand how the neural ...
1
vote
2answers
40 views

How to extract the function being approximated by a neural network?

We all know that in general, a neural network takes in a set of training examples having the form $\{x, f(x)\}$ and it aims to approximate the function $f$ thereby "classifying" $x$ to its correct ...
0
votes
0answers
12 views

How use F1 score in an unbalanced binary classification problem?

I have two trained models (MLP and SVM) that want check on unbalanced binary samples (out of sampl - True samples =3000, False samples = 200). I found that i can use ...
0
votes
0answers
24 views

Area under ROC curve vs. Accuracy in unbalanced sample

I have a binary classification problem with 3000 samples (number of 1 as outputs = 300, number of 0 as outputs = 1700). After balancing database (selecting 300 samples from 0 outputs) I trained the ...
0
votes
1answer
54 views

Neural networks: how can convex optimization produce different weights each time?

I am training a multilayer perceptron with a logistic activation function by backpropagation. The weights are not unique - each time I redo the fit, I get a different set of weights. However the ...
4
votes
1answer
36 views

How is softmax unit derived and what is the implication?

I'm trying to understand why the softmax function is defined as such: $\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$ I understand how this normalizes the data and properly maps to some ...
0
votes
0answers
20 views

What to expect when training deep neural networks with increasing capacity?

I have a question with regards to the order in which to go about when trying different deep neural network architectures for a task. Suppose I trained a model with $|P_1|$ parameters, and noted down ...
0
votes
0answers
23 views

Obtaining Hopfield network weights from energy function

I'm trying to come up to speed on Hopfield networks. When applied to solving the traveling salesman problem, for example, papers present their energy function, but never explain how they derive the ...
0
votes
0answers
69 views

Random Forest accuracy 0.98, is it too much?

I am using about 256 predictors and target is sales. I am using a software called Alteryx which is R based. I have tried to run Random Forest, Spline model and Neural nets on same data. I used ...
0
votes
0answers
11 views

Robot localization and position prediction

The problem I am working on is related to robot localization and position prediction in the future. Given a simple video of a robot bouncing around in a wooden box and a mapping of the coordinates ...
0
votes
0answers
49 views

How are tensors used in neural networks?

I'm new to learning neural networks and am trying to understand the role of tensors in them. I am trying to use some neural network libraries, but they are asking me for the dimensions. Could any one ...
0
votes
0answers
10 views

Using GIS and other metadata as additional features in Image recognition CNN

I'm wondering about including additional features into a CNN model that is built to classify images. Has there been much work or research in adding more traditional features into such a model ...
1
vote
1answer
33 views

Confusion Matrix Matlab has a 0% for a class during training

When I train Multi-Layered Perceptron using Matlab for 6 classes, for one of the classes I get a 0% in GREEN in the confusion matrix. This leads to to a high error percent for the overall training, ...
0
votes
1answer
43 views

Can I use ReLU in autoencoder as activation function?

When we discuss about autenocder in neural network, most people will use sigmoid as the activation function. Can we use ReLU instead? (Since ReLU has no limit on the upper bound, basically meaning ...
0
votes
0answers
43 views

How can we map arbitrary-length high-dimensional sequences to arbitrary-length high-dimensional sequences using recurrent neural-networks?

The aim is to synthesize sequence in one domain given other. examples: Text to Speech using RNN. Lets say I have an arbitrary text as sequence and I want output speech of corresponding word but ...
1
vote
0answers
28 views

Validating Neural Network

I have a relatively large set of labelled data which I am using to verify how well-trained a neural-network I wrote is. The network uses a competitive learning technique. It has N output neurons, each ...
2
votes
0answers
17 views

Is Cross entropy cost function for neural network convex?

My teacher proved that 2nd derivate of cross-entropy is always positive, so that the cost function of neural networks using cross entropy is convex. Is this true? I'm quite confuse about this because ...
0
votes
1answer
32 views

Training a neural network with uniform random inputs

I'm playing with a little prediction project using a multi-layer perception (MLP) with robust back propagation. I have a variety of variables which correlate to the single output I'm attempting to ...
1
vote
0answers
27 views

Sensitivity Analysis used in SPSS Neural Network Package

What is the sensitivity analysis used in SPSS Neural Network's independent variable importance calculation? The explanation provided by SPSS is very vague. Is it the same as the semipartial ...
0
votes
1answer
14 views

MLP with 2 outputs vs 2 MLPs with single outputs for nonlinear regression

Assume i want to apply nonlinear regression to two output variables with multilayer perceptrons. Is there difference between using a MLP for each regression with single output and using a single MLP ...
2
votes
1answer
37 views

nnet function in R

I have 7 input variables,2 hidden neuron and 1 output variable.the train sample are 50. I used nnet() function in R to train my network but it returns 187-2-50 ...
0
votes
0answers
24 views

Is a neural network's output a function composition of previous layers' activation functions?

I am reading Hornik's paper which shows that multi-layer neural networks are universal approximators. In it he has the following definition: I uploaded an image rather than attempt at rewriting it ...
1
vote
0answers
21 views

advices about train set and validation set

I know that there should be three sets of tests for supervised learning that are: train validation test I have read that for example in the case of NN in the train phase one chooses the weights ...
1
vote
1answer
18 views

Is my interpretation of the numerical gradient versus network output correct?

I have implemented a neural network with back propagation using a sigmoid activation function. To validate the functionality of my code, I am estimating the gradient of my function using the ...
0
votes
1answer
37 views

What is obtained from the product of a probability and a log probability ratio? [closed]

I'm looking at the commonly used artificial neural network model that has nodes and connections. Quick refresher A connection has a source and target node, and a weight. The output of the source ...
0
votes
1answer
50 views

What is the mathematical underpinning of feedforward artificial neural network?

For a school project, I have implemented a 3 layer feedforward ANN with an RBF activation function that can be used to distinguish between different types of signals. I have a demonstration coming up ...
0
votes
0answers
24 views

normalizing data for neural network

I'm working on a neural network with back propagation for indoor localization. The input of the neural network is Received Signal Strengths (RSSs) and the output is a coordinate (x,y). I have ...
0
votes
0answers
31 views

trouble with understanding neural network

Can anyone give an explanation for a page 422 from the The Elements of Statistical Learning. I couldn't understand the meaning of 'the least constrained model.' Paragraph and picture is shown as ...
0
votes
1answer
26 views

Prediction vs. Classification in neural networks

I am learning the backpropagtion algorithm, and would like to clarify some concepts. Suppose my training data set consists of 20-dimensional bit strings that are classified into 5 different classes. ...
3
votes
1answer
31 views

How to make a trained neural network “forget” an instance?

I am using neural networks for predicting the behavior of a dynamic system. A neural network is trained online using snapshots from the system's past. The system changes its state at irregular ...
2
votes
1answer
17 views

Contradictory input/output pairs when training neural network?

I think this question can probably be generalized away from neural networks, bu there goes: How should we handle possibly contradictory data? Suppose the neural networks maps n-bit strings to a bit. ...
0
votes
0answers
45 views

using Cross Validation in matlab with neural networks

I want to make a cross validation on neural network, I tried to pass the labels to crossval function, with the help of ...
0
votes
0answers
14 views

Feed-Forward Neural Networks Query

Is there a way to generate and ideal input vectors given an observed output vector in a trained network. In a lot of Autoencoder tutorials it is shown how to visualize 1 unit. Can this be extended to ...
1
vote
2answers
63 views

tanh vs. sigmoid in neural net

I apologize in advance for the fact that I'm still coming up to speed on this. I'm trying to understand the pros and cons of using tanh (map -1 to 1) vs. sigmoid (map 0 to 1) for my neuron activation ...
4
votes
3answers
109 views

(Feed-Forward) Neural Networks keep converging to mean

I'm having an interesting dilemma with the neuralnet and nnet packages in R. I recently ...
0
votes
1answer
19 views

LSTM forgetting dependencies

How does a LSTM network know when is a good time to forget the dependencies it has learned? I understand that it forgets when the value of forget gate is close to zero. But how does it know when to ...
1
vote
0answers
21 views

Trying to impelement IRPROP+

I'm stuck I tried 3 times to setup IPROP+. I figured IPROP+ was the most highly rated of the three found here http://heatonresearch.com/wiki/RPROP Problem is... my training doesn't seem to work. ...
0
votes
1answer
45 views

Using Adaptive Linear Neurons (Adalines) and Perceptrons for 0-1 class problems

I am wondering how to adjust the Adaline algorithm to classify the classes 0 and 1 instead of -1 and 1. I found a section in Neural Networks and Statistical Learning by Ke-Lin Du, M. N. S. Swamy ...
1
vote
1answer
123 views

Deep neural nets, RELU's removing non-linearity?

are RELU (Rectified Linear Units) activation functions considered non-linear? They are linear when the input is > 0 and from my understanding to unlock the representative power of deep networks ...