# Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

100 questions
6answers
14k views

### What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
3answers
11k views

1answer
2k views

### What is the correct way of calculating Rectifier Linear and MaxOut functions?

If I have a artificial neuron with 2 inputs: input 1 = 0.7 & weight = 0.7 input 2 = 0.3 & weight = 0.3 If I use a Rectifier Linear (ReLU) as activation ...
1answer
492 views

### Machine learning: intuition behind perceptron learning algorithm

Given features $x_1...x_n$, weights $w_1...w_n$, calculated output $y = W^T \cdot X$, and actual output $\hat{y}$, the perceptron learning algorithm changes the weights after each iteration as follows:...
1answer
2k views

### What are the best R packages for a classification problem with use of Neural networks [closed]

Surfing on the internet shows me that there are a lot of different packages and functions which can be used to train neural networks via R. packages such as 'RSNNS', 'nnet','neuralnet', etc. I'm ...
2answers
1k views

### Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
2answers
2k views

### How to Score an MLP Classifier

I am self-teaching Machine Learning, so excuse me if this is a stupid question. I am trying to understand MLP Classifiers, but would like to know what the best way to create a score is. e.g. ...
1answer
238 views

### Why cannot my sigmoid classify a linearly separable set with 100% accuracy?

I recently finished my implementation of a feedforward multilayer ANN in Julia. I train it using basic gradient descent with no additions (no regularization, no momentum, no decay, no anything, just ...
1answer
1k views

### Calculation of decision boundaries with Perceptron

here i will train perceptron and plot decision boundaries (target is generated so I am sure that it is lineary separable). For sake of the example, there is no bias. ...
1answer
850 views

### Resilient Propagation: How to choose between RPROP+, RPROP-, iRPROP+, and iPROP-

I am using ENCOG to implement a Perceptron network. One of the easiest back-propagation (gradient descent) algorithms to use is the Resilient Propagation algorithm. There are four variants for ...
0answers
114 views

### Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
0answers
74 views

### Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
0answers
355 views

### Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
0answers
1k views

### Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
0answers
69 views

### Perceptron trained on time series always predicting the same answer [duplicate]

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
0answers
157 views

### Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
0answers
411 views

### Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm

Recall the perceptron algorithm: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$ ...
2answers
370 views

### Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
2answers
2k views

### Perceptron overfitting?

I'm trying to judge the performance of my perceptron linear discriminant. In one instance I'm training on a sample size of 150 and on another I'm training on a sample size of 1500. I test both of ...
1answer
247 views

1answer
462 views

### Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
1answer
904 views

### Intuition on upper bound of the number of mistakes of the perceptron algorithm and how to classify different data sets as “easier” or “harder”

I was trying to understand the following result more intuitively (for linearly separable data): $$k \leq \frac{R^2}{\gamma_g^2}$$ where: k = is the number of mistakes the perceptron algorithm does ...
1answer
40 views

### Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
0answers
38 views

### Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
0answers
112 views

### Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
1answer
226 views

### Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...