Neural networks traditionally refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks (ANN), which are composed of artificial neurons or nodes - programming constructs that mimic the properties of biological neurons. ANN are ...

learn more… | top users | synonyms

0
votes
0answers
2 views

Neural network and dynamical system

Dynamical systemsare those whose evolution can be described by a rule, evolves with time and is deterministic. In this context can I say that Neural networks have a rule of evolution which is the ...
0
votes
0answers
7 views

What is R squared for a neural network and what does it signify?

I calculated R square for my neural network based on a formula I found somewhere, which goes something like: http://i.stack.imgur.com/DojZC.png It should be something around 0.98-0.99. But, when I ...
0
votes
0answers
28 views

Algorithm for online handwriting recognition

Is there any specific algorithm for online handwriting recognition? The algorithm should recognize non-cursive and cursive handwriting. I know there is already a similar post on stackoverflow.com, ...
0
votes
1answer
13 views

How is the “classification error rate” of an artificial neural network calculated?

Frequently I see artificial neural networks compared by their "classification error rates" or "error rates", particularly for multi-class problems like CIFAR-10. What does this error rate actually ...
0
votes
0answers
21 views

How to understand this objective function in deep learning

I'm going through Christopher Manning's tutorial from NAACL 2013 "Deep Learning for NLP (without Magic)" and he gets to the point where he's showing how to do unsupervised pre-training. He's saying ...
0
votes
0answers
11 views

Activation Function [on hold]

In Phd research about ANN cost model I found this log sigmoid function $f(x_j )=\frac{1}{1 + e^{-x_j}}$ equivalence to this one $f(x_j )=\frac{1}{1+e^{-(b+tanh(x_j)}}$ Is there any ...
1
vote
1answer
49 views

Does the vanishing gradient in RNNs present a problem?

One of the often cited issues in RNN training is the vanishing gradient problem [1,2,3,4]. However, I came across several papers by Anton Maximilian Schaefer, Steffen Udluft and Hans-Georg Zimmermann ...
1
vote
2answers
64 views

Which classifiers work well with unbalanced data?

I have a binary classification problem which is very unbalanced - it can have 98% of data from one class. Which classifiers work well with this sort of data? I have an unlimited supply of training ...
0
votes
0answers
16 views

Matlab GUI background color changing to black when using parallel computing for MLP and SVM [closed]

I designed a GUI in Matlab that uses parallel computing in loops for accelerating speed. When I disable parallel computing everything Is normal but when I activate it background color of my GUI and ...
0
votes
0answers
32 views

10 fold cross validation model in weka

Trying to build a specific Neural Network arcitecture and testing it using 10 fold cross validation of a dataset. Now building the model is a tedious job and Weka expects me to make it 10 times for ...
0
votes
2answers
27 views

Good machine learning models for confusable categories

I'm using the word confusable to represent similar looking glyphs in text. I'm building an optical character recognition tool with the primary goal of experimenting with machine learning – especially ...
0
votes
0answers
29 views

Arima model - multi step forecast

The following code shows a forecast of the next 24 hours of my electricity prices with two exogenous variables. My problem is, that I don't know how to build a forecast for the next 3 days or more ...
0
votes
0answers
15 views

Forensics in wireless networks, anomaly detection and beyond?

first i'de like to apologize if this is not the right place. Next year i'm gonna be working on my final project in computer security, i have to build a wireless forensics tool that can analyse a data ...
0
votes
1answer
37 views

Neural networks - are local minima bad?

I hear a lot about local minima for neural networks. I understand the theory behind it - but if my neural network finds weights in a local minimum, is that a bad thing? I understand that finding ...
1
vote
0answers
25 views

Neural networks creates negative output

I am using a simple feedforward neural network in MATLAB to predict the output for inputs in the range [1e-5, 0.3]. (These are the activations of another network.) I am using a sigmoid function for ...
0
votes
0answers
10 views

Basic Question: how does feed-forward neural network solve regression?

This is a fairly basic question but I can't seem to find an answer on the net (perhaps I'm searching the wrong things). Regression is trying to predict continuous outputs. Since a neural network uses ...
1
vote
1answer
36 views

Normalizing Vs. Scaling

Are the concepts of normalizing and scaling of data in conflict with each other? I am adding weights to my features, I have tried normalizing the weights and it didn't make any difference in the ...
0
votes
1answer
25 views

How to : a brief intro to scaling and rescaling data ( inputs) for supervised learning algorithms

I understand the concept of scaling and that it improves results in SVM's and NN's. however I would like to find somewhere where is is explained, in easy "layman's terms" terms. of how it is done. I ...
1
vote
1answer
22 views

Scaling in SVM (why and how to , plus references)

Hi I know why feature scaling is preferred in SVM, I have two questions: 1-does anyone know of legit articles of books explaining it. I am writing my thesis and I need references. It doesnt have to be ...
0
votes
0answers
21 views

How to avoid NaN in using ReLU + Cross-Entropy?

ReLU has a range of [0, +Inf). So, when it comes an activation value z=0/1 produced by ReLU or softplus, the loss value computed by cross-entropy : loss = -(x*ln(z)+(1-x)*ln(1-z)) will turn to NaN. As ...
0
votes
1answer
39 views

test validation versus k-fold cross validation

I am attempting to use a neural network, after using other machine learning algorithms. I am using the RSNNS package (I am willing to use / evaluate other packages) that's part of R. I would like to ...
0
votes
1answer
37 views

Question about Neural Network Training

What is the benefit of optimizing our neural network error function using back prop rather than just using gradient or stochastic gradient descent directly on the error function? How come we don't ...
1
vote
0answers
13 views

Convolutional networks: how to deal with duplicate filters?

I have to train a convolutional network. When I visualize its conv filter matrices from a layer, many filters are quite similar, almost equal. 1) Don't you know a common way to detect couples of ...
0
votes
1answer
24 views

Cost function of neural network is NON-CONVEX?

The cost function of neural network is $J(W,b)$, and it is claimed to be non-convex. I don't quite understand why it's that way, since as I see that it's quite similar to the cost function of Logistic ...
0
votes
0answers
42 views

How do I ensure my implementation of a MLP from Weka is correct?

I have been training MLPs for a binary classification problem in the Weka explorer and now have one with an acceptable level of accuracy. I've written some code to parse the text of the model which ...
1
vote
0answers
31 views

Tips for training dropout neural network

I use NN for my mini project research, and I found out the newest trick for feed forward NN is using dropout for regularization instead of L1/L2 norm and rectified linear unit as an activation ...
1
vote
0answers
22 views

Bayesian Perceptron - how can I generate many different perceptrons?

I am going to implement the Bayesian version of a perceptron that I read in the Statistical Mechanics of learning, by Engel-Van Den Broeck. The idea to improve the performance is to use many Gibbs ...
0
votes
2answers
70 views

If data has no noise can a neural network achieve 100% accuracy?

If you have a function that you want a neural network to learn, and you have sufficient examples of data with the right coverage, and there is no noise in the data, then is it reasonable to expect to ...
0
votes
1answer
17 views

How to speed up the training in neural network when mini-batch training is used?

Can anyone give me some ideas on possible techniques to speed up the training process of multilayer artificial neural network if the training involves mini-batch? So far, I understand that stochastic ...
0
votes
0answers
16 views

Do we multiply the derivative of the sigmoid function when computing the errors of the weights connecting the last hidden layer to the output layer?

See I was under the impression that when we are calculating the error with respect to the weights connecting the last hidden layer with the output layer we are suppose to get the error of the output ...
0
votes
0answers
38 views

Analyzing neuralnet functions in R

The 'neuralnet' package in R allows us to use neural network algorithm with backpropagation. I want to use the function for prediction. I saw a tutorial on neuralnet in which predictions on the iris ...
0
votes
2answers
77 views

Logistic Regression Vs Neural Networks

This is what I have understood about neural networks. There are $L$ layers from input to output (including both) and the equations as follows. $a(2) = \theta(1) x$ $a(3) = \theta(2)a(2)$ . . . ...
0
votes
0answers
13 views

Neural nets method? Am i doing something wrong?

To set the scene - i am using neural nets in SAS program i have ~ 1000 points variables have been selected prior to this on a combination of human knowledge and variable recombination (PCA etc) I am ...
0
votes
0answers
27 views

What are some linearly inseparable data sets for testing support vector machines and artificial neural networks?

I am looking for some linearly inseparable classification problems and nonlinear regression problems. What are some public data sets?
0
votes
1answer
40 views

How to encode categorical variables for neural networks

In regression, you encode a categorical variable with n possible values using n-1 indicator variables. How about for neural networks?
1
vote
1answer
54 views

Image processing with neural network

I am trying to learn how Neural Network works on image recognition.I am confused that how neural network that how i will give input.My defination is find(track) object in squence of images(particular ...
0
votes
1answer
35 views

Question about normalize/scale data before using neuralnet

I have read several threads about the issue on same outputs after people fitting a neural network model with R neuralnet. Posted Solution is to normalize or scale the data before fitting model. Since ...
0
votes
0answers
14 views

Measuring NN saturation: calculating probabilities

I am trying to devise a measurement of neural network saturation for my NN saturation study. Some background: saturation occurs when a hidden neuron outputs values close to the extremes (usually 0 and ...
0
votes
0answers
18 views

How Sensitive Are Neural Networks?

CrossPost: https://stackoverflow.com/questions/24301472/how-sensitive-are-ff-neural-networks I am aware of pruning, and am not sure if it removes the actual neuron or makes its weight zero, but I am ...
1
vote
0answers
41 views

How to deal with neural networks when the data is unbalanced (but unlimited)?

If you assume that you have a plentiful/unlimited supply of training data, and you are using a neural network for classification, how do you deal with a situation in which a very high proportion of ...
0
votes
0answers
18 views

Good way to use adaptive learning rates in neural network

Adaptive learning rates means using different learning rate for different weight in neural network. Except for the emperical method which updates these learning rates based on consistency in gradient, ...
0
votes
0answers
20 views

Sparse ELM vs SVM

What's the difference between SVM and Sparse Extreme Learning Machine with Gaussian kernel proposed in the following paper:http://www.ntu.edu.sg/home/egbhuang/pdf/Sparse-ELM-IEEE-T-Cybernetics.pdf As ...
2
votes
1answer
40 views

Having a Neural Network recreate what it's learned

I've created a basic Neural Network that learns from basic information and can verify whether or not a piece of information matches it's parameters from a match percentage. Conceptually however, I ...
0
votes
0answers
13 views

LMS cost function vs cross entropy cost function in neural networks

What is difference between using various cost functions: LMS,Cross entropy in neural networks? All of them have same derivative w.r.t final activation and hence all the gradients are still gonna ...
0
votes
1answer
103 views

tanh activation function vs sigmoid activation function

tanh activation function is nothing but 2*sigmoid - 1. Does it really matter between using those two activation functions. Which function is better in which cases?
1
vote
0answers
32 views

Finding criteria for a household financial budget falsification

I’m working on a financial problem about budget of households. Households in a state fill a form about their net budget in every year and our insurance company investigate their financial status and ...
0
votes
1answer
38 views

Caret retraining of neural network after finding optimal parameters?

I am applying a neural network and logistic regression to a classification problem. In order to evaluate the performance of the two classifiers I'm using 5-fold cross-validation (roughly 800 samples ...
2
votes
1answer
96 views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...
0
votes
0answers
21 views

Time and space complexity of Deep Belief Nets (DBN)

What is the time and space(memory) complexity of DBNs? given d:number of dimensions(attributes), n:number of records, and l:number of hidden layers.
1
vote
0answers
21 views

How can you use HMMs and ANNs for on-line handwriting recognition?

I've asked this question on cs.stackexchange before. It has a 20-hours remaining bounty there. On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to ...