# Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

100 questions
14k views

### What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
11k views

2k views

### What is the correct way of calculating Rectifier Linear and MaxOut functions?

If I have a artificial neuron with 2 inputs: input 1 = 0.7 & weight = 0.7 input 2 = 0.3 & weight = 0.3 If I use a Rectifier Linear (ReLU) as activation ...
492 views

### Machine learning: intuition behind perceptron learning algorithm

Given features $x_1...x_n$, weights $w_1...w_n$, calculated output $y = W^T \cdot X$, and actual output $\hat{y}$, the perceptron learning algorithm changes the weights after each iteration as follows:...
2k views

### What are the best R packages for a classification problem with use of Neural networks [closed]

Surfing on the internet shows me that there are a lot of different packages and functions which can be used to train neural networks via R. packages such as 'RSNNS', 'nnet','neuralnet', etc. I'm ...
1k views

### Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
2k views

### How to Score an MLP Classifier

I am self-teaching Machine Learning, so excuse me if this is a stupid question. I am trying to understand MLP Classifiers, but would like to know what the best way to create a score is. e.g. ...
238 views

### Why cannot my sigmoid classify a linearly separable set with 100% accuracy?

I recently finished my implementation of a feedforward multilayer ANN in Julia. I train it using basic gradient descent with no additions (no regularization, no momentum, no decay, no anything, just ...
1k views

### Calculation of decision boundaries with Perceptron

here i will train perceptron and plot decision boundaries (target is generated so I am sure that it is lineary separable). For sake of the example, there is no bias. ...
850 views

### Resilient Propagation: How to choose between RPROP+, RPROP-, iRPROP+, and iPROP-

I am using ENCOG to implement a Perceptron network. One of the easiest back-propagation (gradient descent) algorithms to use is the Resilient Propagation algorithm. There are four variants for ...
114 views

### Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
74 views

### Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
355 views

### Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
1k views

### Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
69 views

### Perceptron trained on time series always predicting the same answer [duplicate]

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
157 views

### Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
411 views

### Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm

Recall the perceptron algorithm: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$ ...
370 views

### Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
2k views

### Perceptron overfitting?

I'm trying to judge the performance of my perceptron linear discriminant. In one instance I'm training on a sample size of 150 and on another I'm training on a sample size of 1500. I test both of ...
247 views

462 views

### Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
904 views

### Intuition on upper bound of the number of mistakes of the perceptron algorithm and how to classify different data sets as “easier” or “harder”

I was trying to understand the following result more intuitively (for linearly separable data): $$k \leq \frac{R^2}{\gamma_g^2}$$ where: k = is the number of mistakes the perceptron algorithm does ...
40 views

### Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
38 views

### Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...