Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

Filter by
Sorted by
Tagged with
3
votes
1answer
751 views

Neural Networks - Difference between one dimensional layer vs multi-dimensional layer

Please take a look at these Neural Network architectures: I can understand the architecture of Hidden Layer in Net-2 : you add 12 Neurons to your hidden layer... What I can't understand is the 2-...
8
votes
3answers
10k views

Difference between MLP(Multi-layer Perceptron) and Neural Networks?

I am wondering about the differences. Based on my understanding, MLP is one kind of neural networks, where the activation function is sigmoid, and error term is cross-entropy(logistics) error. Looking ...
2
votes
1answer
285 views

Why cannot my sigmoid classify a linearly separable set with 100% accuracy?

I recently finished my implementation of a feedforward multilayer ANN in Julia. I train it using basic gradient descent with no additions (no regularization, no momentum, no decay, no anything, just ...
2
votes
1answer
2k views

Calculation of decision boundaries with Perceptron

here i will train perceptron and plot decision boundaries (target is generated so I am sure that it is lineary separable). For sake of the example, there is no bias. ...
1
vote
1answer
317 views

Single Layer Perceptron Question

We have these two mappings: 1) 000→0 001→0 010→0 011→1 100→0 101→1 110→1 111→1 2) ...
4
votes
1answer
686 views

Machine learning: intuition behind perceptron learning algorithm

Given features $x_1...x_n$, weights $w_1...w_n$, calculated output $y = W^T \cdot X$, and actual output $\hat{y}$, the perceptron learning algorithm changes the weights after each iteration as follows:...
2
votes
0answers
165 views

Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
0
votes
1answer
1k views

SLP vs. MLP: Is my data linearly separable?

I implemented an artificial neural network using scikit neuralnetwork. As default configuration for my classification task I am using 10730 Datsets x 115 Features 1 Hidden Layer with 61 neurons 7 ...
7
votes
1answer
7k views

When to use RBF networks instead of multilayer perceptron?

I understand that a radial basis function neural network (RBF) usually has 1 hidden layer, and it differs from a multi-layer perceptron (MLP) via its activation and combination functions among other ...
1
vote
0answers
167 views

How to combine perceptron weights and probabilities

I am working on machine transliteration using structured perceptron to learn the model parameters at word level. I am using beam-search to extract the top $n$ transliterations of a word and would like ...
5
votes
2answers
2k views

How to make the conclustion that VC-dimension for hyperplane in $\mathbb{R^{3}}$ is strictly less than $\mathbb{R^{4}}$?

Given four points in $\mathbb{R^{3}}$ real space, $S = \{(1, 1, 1), (1, 1, -1), (-1, -1, 1), (-1, -1, -1)\} \in \mathbb{R^{3}}$. Will these four points be shattered by a hyperplane in $\mathbb{R^{3}}...
2
votes
1answer
942 views

Resilient Propagation: How to choose between RPROP+, RPROP-, iRPROP+, and iPROP-

I am using ENCOG to implement a Perceptron network. One of the easiest back-propagation (gradient descent) algorithms to use is the Resilient Propagation algorithm. There are four variants for ...
0
votes
1answer
114 views

Why was it ever thought multi layer perceptrons couldn't implement XOR functions?

As I understand it the Perceptrons books helped start the "AI Winter" because the authors claimed that multilayer perceptrons couldn't implement non linearly separable functions like XOR. However it ...
1
vote
0answers
153 views

Neural Networks — How to design for multiple outputs [duplicate]

Given a multilayer perceptron to be used to separate 2 classes. One has 2 design options. 1 -- Use one output node -- where one class is trained to give an output equal to zero, and the other class ...
1
vote
1answer
684 views

Is Weight Matrix Wij from neuron j to i or from i to j?

I have a question about the Weight matrix from a Neural network. I see different notations for this. I see some publications (e.g. This stanford page) with: Notation 1: $W^{(l)}_{ij}$ denotes the ...
0
votes
1answer
551 views

Bias input in neural network

Does bias input work like constant value in linear regression? and if bias input is not used then resulting boundary will always pass through origin? Thanks in advance.
0
votes
1answer
35 views

How to define user efficiency time

I'm making a user study. Users had to perform the same task 10 times. I've recorded the completion times of each task. Now I need to calculate the efficiency time. As you can imagine the user, when ...
11
votes
1answer
3k views

How to kernelize a simple perceptron?

Classification problems with nonlinear boundaries cannot be solved by a simple perceptron. The following R code is for illustrative purposes and is based on this example in Python): ...
3
votes
1answer
2k views

What is the correct way of calculating Rectifier Linear and MaxOut functions?

If I have a artificial neuron with 2 inputs: input 1 = 0.7 & weight = 0.7 input 2 = 0.3 & weight = 0.3 If I use a Rectifier Linear (ReLU) as activation ...
2
votes
1answer
1k views

Is the decision boundary of voted/average perceptron linear like basic perceptron?

Is the decision boundary of voted/average perceptron linear like basic perceptron? How can you justify that? It's related to solve binary classification problems in Machine Learning context.
1
vote
0answers
401 views

Why use a restricted Boltzmann machine rather than a multi-layer perceptron?

I'm trying to understand the difference between a restricted Boltzmann machine (RBM), and a feed-forward neural network (NN). I know that an RBM is a generative model, where the idea is to reconstruct ...
1
vote
0answers
280 views

Perceptron Learning Algorithm: what is the probability that the viewed data is linearly separable, after some number of steps?

My understanding is that the Perceptron Learning Algorithm: will not converge if the data is not linearly separable. might take exponentially many iterations, even if the data is linearly separable. ...
0
votes
1answer
56 views

Single artificial neuron easily extendable to neural network

I'm working on implementation of artificial neuron which be extended to neural net. I want do implementation by myself to fully understand how it works. I start with perceptron with threshold ...
32
votes
6answers
24k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
1
vote
1answer
497 views

Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
2
votes
1answer
2k views

What are the best R packages for a classification problem with use of Neural networks [closed]

Surfing on the internet shows me that there are a lot of different packages and functions which can be used to train neural networks via R. packages such as 'RSNNS', 'nnet','neuralnet', etc. I'm ...
0
votes
1answer
291 views

Neural networks: how can convex optimization produce different weights each time?

I am training a multilayer perceptron with a logistic activation function by backpropagation. The weights are not unique - each time I redo the fit, I get a different set of weights. However the ...
0
votes
1answer
2k views

Using Adaptive Linear Neurons (Adalines) and Perceptrons for 0-1 class problems

I am wondering how to adjust the Adaline algorithm to classify the classes 0 and 1 instead of -1 and 1. I found a section in Neural Networks and Statistical Learning by Ke-Lin Du, M. N. S. Swamy ...
2
votes
0answers
670 views

Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm

Recall the perceptron algorithm: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$ ...
21
votes
3answers
13k views

From the Perceptron rule to Gradient Descent: How are Perceptrons with a sigmoid activation function different from Logistic Regression?

Essentially, my question is that in multilayer Perceptrons, perceptrons are used with a sigmoid activation function. So that in the update rule $\hat{y}$ is calculated as $$\hat{y} = \frac{1}{1+\exp(...
15
votes
1answer
14k views

Clarification about Perceptron Rule vs. Gradient Descent vs. Stochastic Gradient Descent implementation

I experimented a little bit with different Perceptron implementations and want to make sure if I understand the "iterations" correctly. Rosenblatt's original perceptron rule As far as I understand, ...
4
votes
2answers
2k views

Formula for decision boundary of a classifier (in order to visualize it)

I'm confused on how to plot decision boundary for classifiers. For example, i'm working with perceptron. So, the formula for decision boundary(if I understand this correctly) is ...
14
votes
4answers
16k views

What is the difference between a neural network and a perceptron?

Is there any difference between the terms "neural network" and "perceptron"?
1
vote
2answers
3k views

Perceptron overfitting?

I'm trying to judge the performance of my perceptron linear discriminant. In one instance I'm training on a sample size of 150 and on another I'm training on a sample size of 1500. I test both of ...
1
vote
1answer
1k views

Intuition on upper bound of the number of mistakes of the perceptron algorithm and how to classify different data sets as “easier” or “harder”

I was trying to understand the following result more intuitively (for linearly separable data): $$ k \leq \frac{R^2}{\gamma_g^2}$$ where: k = is the number of mistakes the perceptron algorithm does ...
4
votes
0answers
1k views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
11
votes
2answers
29k views

Decision boundary plot for a perceptron

I am trying to plot the decision boundary of a perceptron algorithm and I am really confused about a few things. My input instances are in the form $[(x_{1},x_{2}), y]$, basically a 2D input instance (...

1 2
3