1
vote
0answers
10 views

How do we get/define filters in convolutional neural networks?

How do i obtain filters from convulutional neural network(CNN)? My idea is something like this: Do random images of the input images (28x28) and get random patches (8x8). Then use autoencoders to ...
1
vote
1answer
30 views

Does Deep network (e.g. # of hidden layer=2) always better than shallow network (i.e. # of hidden layer=1)?

I attempted to build a deep network (e.g. deep autoencoder) for some object classification, my result showed that the deep networks is worst than shallow network. However, from what I have read from ...
0
votes
0answers
16 views

Graphically, how does the non-linear activation function project the input onto the classification space?

I am finding a very hard time to visualize how the activation function actually manages to classify non-linearly separable training data sets. Why does the activation function (e.g tanh function) ...
1
vote
1answer
23 views

How to apply the output layer function in a neural network

I am implementing a Neural Network in a somewhat different fashion. I train my neural network locally using a small subset, and export the weights. My goal is to test the neural network in a ...
1
vote
2answers
26 views

What does “permutation invariant” mean?

I have seen a term "permutation invariant" version of the MNIST digit recognition task. What does it mean?
1
vote
2answers
52 views

What're the differences between PCA and autoencoder?

Both PCA and autoencoder can do demension reduction, so what are the difference between them? In what situation I should use one over another?
2
votes
1answer
22 views

What does pre-training mean in deep autoencoder?

I am confused by the term "pre-training". What does it mean in deep autoencoder? And how does it help improving the performance of autoencoder? (I know this term comes from Hinton 2006's paper: ...
0
votes
0answers
18 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron ...
0
votes
0answers
19 views

Do we need to do sigmoid after pooling in convolutional neural network?

I understand we need to do sigmoid transformation after convolution step in building convolutional neural network(CNN). Do we need that also after pooling step? Like: Let assume: CL = convolution ...
0
votes
0answers
25 views

What is the correct architecture for convolutional neural network?

I have seen several different architectures for convolutional neural network (CNN). I am confused which one is the standard and how do I decide what to use. I am not confused by the number of layers ...
0
votes
1answer
29 views

In Convolutional neural network, what does fully-connected layer mean?

There are convolution layers, pooling layers, and possibly a classifier layer (e.g. softmax layer) in convolutional neural network(CNN). I heard that there is also a fully-connected layer, what is ...
1
vote
0answers
29 views

Can a perceptron be modified so as to converge with non-linearly separable data?

Normally, a perceptron will converge provided data are linearly separable. Now if we select a small number of examples at random and flip their labels to make the dataset non-separable. How can we ...
6
votes
2answers
75 views

What are the differences between sparse coding and autoencoder?

Sparse coding is defined as learning an over-complete set of basis vectors to represent input vectors (<-- why do we want this) . What are the differences between sparse coding and autoencoder? ...
1
vote
0answers
31 views

How can I make sure that an LDA implementation works?

I am currently experimenting with neural nets for classification of on-line handwritten data (hence: not pixels, but time series data). To do so, I use several toolkits (internal development of my ...
0
votes
0answers
21 views

What is the use of convolution and pooling in convolutional neural network

I am confused the use of convolution and pooling in convolutional neural network(CNN). I know pooling is basically summarizing the adjacent features, which can greatly reduced the features size of an ...
1
vote
0answers
36 views

Performance of algorithms using Jacobian matrix on large data sets

Some ANN learning algorithms like Gauss-Newton, Levenberg-Marquardt require creation of a Jacobian matrix. Having studied this I found out that the Jacobian matrix is huge (unless I misunderstood ...
1
vote
1answer
15 views

Weight Decay in Neural Neural Networks Weight Update and Convergence

I have a neural network (That I created using java) for a class assignment that is working when I do not use any weight decay value, but when I use a value greater than or equal to .001, my accuracy ...
0
votes
0answers
33 views

Using third validation set in Cross Validation?

(Note there's 2 paragraphs of background information before I get to the question) I've got a Neural Network classifier, trained with an EA to classify data. I previously used a holdout framework ...
0
votes
0answers
187 views

In Graph Transformer Networks, which parameters are tuned during back-propagation?

Referring to the milestone paper "Gradient-Based Learning Applied to Document Recognition" of LeCun, Bottou, Bengio and Haffner, which parameters of the graph transformer networks for global training ...
0
votes
0answers
27 views

Neural Networks sigmoid activation with bias updates

I am trying to figure out if I am creating an artificial neural network using the sigmoid activation function and using bias correctly. I want one bias node to input to all hidden nodes with static ...
0
votes
0answers
40 views

What is a convolutional neural network

I have been studying neural networks and I recently found out about deep learning and convolutional neural networks. Can someone give me a newbie introduction to convolutional neural networks, what ...
0
votes
1answer
20 views

Requirements for a valid neural network activation function?

What rules define a valid neural network activation function, excluding biological plausibility? What set of principles do softmax, rectified linear units, hyperbolic tangent, sigmoid, etc. follow? ...
0
votes
1answer
55 views

Logistic Regression, SVM or NN?

Just attended Andrew Ng’s online course on ML and although I’ve understood the methods I seem to be missing the intuition on where to apply them in terms of classification problems. What are the ...
3
votes
2answers
90 views

Newbie to neural networks

Just starting to play around with Neural Networks for fun after playing with some basic linear regression. I am an English teacher so don't have a math background and trying to read a book on this ...
0
votes
0answers
19 views

The ethics of using an optimal multiclass feature set for binary classification

I'm currently trying to find the best feature set/network architecture configuration for a binary classification problem, however to approach it via the usual means of building and testing does not ...
0
votes
3answers
106 views

How to Use Neural Networks to Forecast Time Series Data with Predictor Variables?

I have browsed a lot of topics here, but the ones I see were all about forecasting a single variable, depending on its historical values. Whereas I want to predict a variable, by estimating a ...
1
vote
0answers
21 views

Applying autoencoders for dimensionality reduction in audio: Why does this create a low-pass effect? [closed]

I've been playing around with framing audio data and training a single-layer autoencoder to find a dimensionality-reduced form (say 128-sample frames to 32-dimension frames). When I test the audio ...
0
votes
0answers
22 views

auto-steering using neural networks

I was hoping if anyone could point me in right direction, I want to implement a neural network that could steer an autonomous car, I have implemented basic classification problems before using single ...
1
vote
0answers
41 views

Neural Networks and Picture Recognition

I have spent a bunch of time looking at this series of videos (Neural Network Tutorial), by Ryan Harris: https://www.youtube.com/watch?v=Q_5B3GuWPCc&index=41&list=PL29C61214F2146796 I am ...
1
vote
0answers
46 views

How to give an input when you are using Machine Learning method in R

I am new to R and machine learning algorithms. I have basic knowledge of different machine learning algorithms. I have four years of daily sales data.I am trying to predict sales using Support Vector ...
0
votes
0answers
25 views

Custom Neural network in matlab

I am trying to make a custom Neural network structure using 'network' command. I am a little confused.Can we change the connections between individual neurons?Like it is possible in weka?
0
votes
0answers
55 views

MATLAB interperetation of Neural networks

I am new to Neural networks and I am trying to build a custom neural network using the NN toolbox in MATLAB.I am using the "create custom neural network function". Now, I find the neural network ...
0
votes
0answers
37 views

Neural-Net style pattern recognition with an unknown/varying number of inputs?

Say for example I had a weighted graph such that each node had an associated value. The nodes' values are given by some function of the edge weights and the number of edges as well as the node's ...
0
votes
0answers
55 views

Algorithm for online handwriting recognition

Is there any specific algorithm for online handwriting recognition? The algorithm should recognize non-cursive and cursive handwriting. I know there is already a similar post on stackoverflow.com, ...
1
vote
1answer
131 views

Does the vanishing gradient in RNNs present a problem?

One of the often cited issues in RNN training is the vanishing gradient problem [1,2,3,4]. However, I came across several papers by Anton Maximilian Schaefer, Steffen Udluft and Hans-Georg Zimmermann ...
0
votes
0answers
228 views

10 fold cross validation model in weka

Trying to build a specific Neural Network arcitecture and testing it using 10 fold cross validation of a dataset. Now building the model is a tedious job and Weka expects me to make it 10 times for ...
0
votes
0answers
18 views

Forensics in wireless networks, anomaly detection and beyond?

first i'de like to apologize if this is not the right place. Next year i'm gonna be working on my final project in computer security, i have to build a wireless forensics tool that can analyse a data ...
0
votes
1answer
53 views

How to : a brief intro to scaling and rescaling data ( inputs) for supervised learning algorithms

I understand the concept of scaling and that it improves results in SVM's and NN's. however I would like to find somewhere where is is explained, in easy "layman's terms" terms. of how it is done. I ...
1
vote
0answers
28 views

Bayesian Perceptron - how can I generate many different perceptrons?

I am going to implement the Bayesian version of a perceptron that I read in the Statistical Mechanics of learning, by Engel-Van Den Broeck. The idea to improve the performance is to use many Gibbs ...
2
votes
1answer
84 views

Image processing with neural network

I am trying to learn how Neural Network works on image recognition.I am confused that how neural network that how i will give input.My defination is find(track) object in squence of images(particular ...
0
votes
1answer
85 views

Question about normalize/scale data before using neuralnet

I have read several threads about the issue on same outputs after people fitting a neural network model with R neuralnet. Posted Solution is to normalize or scale the data before fitting model. Since ...
0
votes
0answers
26 views

How Sensitive Are Neural Networks?

CrossPost: https://stackoverflow.com/questions/24301472/how-sensitive-are-ff-neural-networks I am aware of pruning, and am not sure if it removes the actual neuron or makes its weight zero, but I am ...
0
votes
0answers
24 views

Good way to use adaptive learning rates in neural network

Adaptive learning rates means using different learning rate for different weight in neural network. Except for the emperical method which updates these learning rates based on consistency in gradient, ...
0
votes
0answers
36 views

Sparse ELM vs SVM

What's the difference between SVM and Sparse Extreme Learning Machine with Gaussian kernel proposed in the following paper:http://www.ntu.edu.sg/home/egbhuang/pdf/Sparse-ELM-IEEE-T-Cybernetics.pdf As ...
2
votes
1answer
45 views

Having a Neural Network recreate what it's learned

I've created a basic Neural Network that learns from basic information and can verify whether or not a piece of information matches it's parameters from a match percentage. Conceptually however, I ...
0
votes
0answers
13 views

LMS cost function vs cross entropy cost function in neural networks

What is difference between using various cost functions: LMS,Cross entropy in neural networks? All of them have same derivative w.r.t final activation and hence all the gradients are still gonna ...
2
votes
1answer
625 views

tanh activation function vs sigmoid activation function

tanh activation function is nothing but 2*sigmoid - 1. Does it really matter between using those two activation functions. Which function is better in which cases?
0
votes
0answers
32 views

Time and space complexity of Deep Belief Nets (DBN)

What is the time and space(memory) complexity of DBNs? given d:number of dimensions(attributes), n:number of records, and l:number of hidden layers.
0
votes
0answers
31 views

Supervised classification on different time series

I have 300 files, each file has a time series data with a class label(0 or1) for each data point.I want to build a classifier, which can predict the class of a new time series data. How should I ...
0
votes
1answer
75 views

neural network output layer for binary classification

I'm using a neural network for a binary classification problem. Is it better to have one neuron in the output layer or to use two, i.e. one for each class?