Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

5
votes
2answers
8k views

What is the difference between MLP and RBF?

What are the main differences between two types of feedforward networks such as multilayer perceptrons (MLP) and radial basis function (RBF)? What are the fundamental differences between these two ...
0
votes
1answer
15 views

Training Perceptrons with Backprop

Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule? is it ...
0
votes
1answer
18 views

No need for bias term if data is standardised? Linear classification models

For linear classification models, e.g. perceptron, bias term allows to move separating hyperplane away from origin. If data is scattered around the zero does that mean that we don't need bias term?
1
vote
1answer
635 views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
0
votes
0answers
29 views

Z score standardisation vs min-max scaling for feature selection

I am applying l1 norm on the input weights of a single layer MLP. I wanted to know if I should standardize or min-max scale ([0 1] feature scaling) my input data?
1
vote
1answer
226 views

Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...
0
votes
0answers
21 views

Choosing perceptron weights to achieve 0% error

I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
1
vote
1answer
43 views

Perceptron learning for non-linearly separable data

It is well known that perceptron learning will never converge for non-linearly separable data. This means that you cannot fit a hyperplane in any dimensions that would separate the two classes. Is it ...
1
vote
1answer
181 views

Intuition behind neural networks

I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks. I'm following the Caltech video which is excellent https://www.youtube.com/watch?v=Ih5Mr93E-...
1
vote
1answer
40 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
0
votes
1answer
28 views

Neural Network Weight Update

I posted this on Software Engineering and was told that it might be better here and on AI. I'm currently reading Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky (3rd ...
2
votes
0answers
114 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
13
votes
3answers
10k views

Multi-layer perceptron vs deep neural network

This is a question of terminology. Sometimes I see people refer to deep neural networks as "multi-layered perceptrons", why is this? A perceptron, I was taught, is a single layer classifier (or ...
0
votes
0answers
30 views

2-layered neural network with linear and sigmoid

Has a 2-layered neural network where each perceptron is a linear unit (no thresholding) has the same expresiveness as a 2-layered neural network where each perceptron is a sigmoid unit?
4
votes
0answers
770 views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
0
votes
2answers
42 views

No error in the training/validation dataset

Suppose I have a linearly separable dataset, divide into training and validation sets. Will a perceptron learned on the training dataset be guaranteed to have no error on the training dataset and on ...
0
votes
0answers
8 views

Why goal of PLA can ignore the norm of normal vector

Define hyperplane $w*x+b=0$, the goal of PLA(Perceptron Learning Algorithm) is minimizing the distance of misclassified points to the decision boundary, i.e. $$-\frac{1}{||w||}\sum_{i\in M} y_i(w*x_i ...
0
votes
1answer
436 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
27
votes
6answers
14k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
1
vote
1answer
253 views

Derivation of Perceptron weight update formula

I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
0
votes
1answer
48 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
0
votes
1answer
31 views

Neural Networks - Back Propagation and Perceptrons

While studying about neural networks (still on basics - not Deep Learning etc.,) two questions came on my mind. What is the reason for replacing the hard limiter function in the nodes of the ...
0
votes
1answer
24 views

Is Perceptron a data-structure or an algorithm or both?

I found Perceptron in the discussions of (overlapping of disciplines is acknowledged): Neural Network Machine Learning Data Mining, Pattern Recognition Genetic Algorithm In the discussion of Neural ...
0
votes
0answers
15 views

Adaboost - Using Perceptrons

This is for a Image orientation project. I'm trying to implement adaboost with four perceptrons. The training data has four labels 0, 90, 180, 270. Each of the perceptron identifies one of the labels ...
0
votes
0answers
80 views

Training a perceptron on MNIST using whole images yielding perfect results

So I programmed a simple Perceptron algorithm to classify images from the MNIST dataset. My goal is to tell apart what image is a zero and what image is a one. First I trained my algorithm by ...
0
votes
2answers
2k views

What is weights in perceptron

I am just diving into machine learning and started with learning artificial neural networks. So on learning about perceptron I stucked on wording "weights". Is it rate of how much input item matched? ...
0
votes
1answer
19 views

Weight of MLP is larger than 1

I noticed when training MLP that weights of neurons can be larger than 1. Would this have negative effects on the outcome of the network? If yes, how to mitigate this problem?
0
votes
1answer
28 views

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality. Less cryptically put, MLP's are great ...
3
votes
1answer
492 views

Machine learning: intuition behind perceptron learning algorithm

Given features $x_1...x_n$, weights $w_1...w_n$, calculated output $y = W^T \cdot X$, and actual output $\hat{y}$, the perceptron learning algorithm changes the weights after each iteration as follows:...
1
vote
0answers
38 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
7
votes
3answers
7k views

Difference between MLP(Multi-layer Perceptron) and Neural Networks?

I am wondering about the differences. Based on my understanding, MLP is one kind of neural networks, where the activation function is sigmoid, and error term is cross-entropy(logistics) error. Looking ...
1
vote
0answers
112 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
0
votes
1answer
55 views

What is the expression for derivative of the signum function one should use in the BP training method

The back propagation learning method requires knowing of derivatives of activation functions. But what expression one should use for signum activation function $$ \mathrm{Signum}( x ) = \...
1
vote
1answer
905 views

Intuition on upper bound of the number of mistakes of the perceptron algorithm and how to classify different data sets as “easier” or “harder”

I was trying to understand the following result more intuitively (for linearly separable data): $$ k \leq \frac{R^2}{\gamma_g^2}$$ where: k = is the number of mistakes the perceptron algorithm does ...
0
votes
1answer
87 views

Clarification on simple perceptron neural network

I understand the title (and the question itself) is a little generic, but I have some questions that I doubt I can find from google search or studying the topic (yes, I've tried for a while). ...
0
votes
3answers
552 views

Does the adjustment / learning of the weights in Perceptron algorithm depend on the learning rate?

For perceptron algorithm, the output and target values are either $0$ or $1$. Assume output is $y$ and target is $d$. From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can ...
1
vote
1answer
94 views

Would multilayer perceptrons be better than multiple regression?

I am using multiple regression to predict the future value of a time series from several other time series. Would doing this with multilayer perceptrons produce better results than multiple regression?...
1
vote
1answer
80 views

Why do nodes in hidden layer produce different results?

Assuming a simple, fully connected Multilayer Perceptron network with one input layer, one hidden layer with multiple nodes and one output layer. In this case the nodes in hidden layer are ...
0
votes
1answer
133 views

neural network - target data

For the simple AND learning with a perceptron, it is required to have two inputs x1 and x2 and one target data y. Most AND examples (such as in the book "Fundamentals of neural network-fausett") have ...
2
votes
2answers
1k views

Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
0
votes
1answer
66 views

Conceptual question on MLP error calculation

Consider a NN of 2 input neurons, one hidden layer of 4 neurons and one output node. The task is to predict the next sample given two input samples at a time. $m_1$ is the output of the hidden layer, $...
2
votes
0answers
74 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
3
votes
1answer
280 views

MLP - what did author have in mind?

I am currently going through Machine learning and pattern recognition and I have following dilemma: A algorithm for MLP is presented in that way and corresponding python code to equation 4.9 (which ...
1
vote
1answer
799 views

How can a perceptron be used for regression?

I know perceptron is a binary classifier which has a 0/1 output. But in one of my exercises for a Neural Network course, there is a question that asks to implement a linear regression with perceptron. ...
0
votes
2answers
1k views

(Deep) Neural Networks/MLPs: Should I normalize/scale my input features when the units of the features are meaningful?

Until now, I always normalized or standardized my features individually before feeding them into a neural network. But at my current project I have features, which in huge parts have the same unit (US-...
2
votes
0answers
159 views

Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
2
votes
0answers
356 views

Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
10
votes
2answers
22k views

Decision boundary plot for a perceptron

I am trying to plot the decision boundary of a perceptron algorithm and I am really confused about a few things. My input instances are in the form $[(x_{1},x_{2}), y]$, basically a 2D input instance (...
1
vote
2answers
370 views

Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
0
votes
1answer
114 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...