Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

0
votes
0answers
28 views

Perceptron learning algorithm. Gradient descent

Linear classifier: $a(x,w) = sign\langle\,x,w\rangle$, if sign is positive object belongs to $+1$ class and if it is negative to $-1$. When we use gradient descent to train the perceptron, we are ...
0
votes
0answers
13 views

Z score standardisation vs min-max scaling for feature selection

I am applying l1 norm on the input weights of a single layer MLP. I wanted to know if I should standardize or min-max scale ([0 1] feature scaling) my input data?
0
votes
0answers
21 views

Choosing perceptron weights to achieve 0% error

I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
1
vote
1answer
25 views

Perceptron learning for non-linearly separable data

It is well known that perceptron learning will never converge for non-linearly separable data. This means that you cannot fit a hyperplane in any dimensions that would separate the two classes. Is it ...
1
vote
1answer
38 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
0
votes
1answer
21 views

Neural Network Weight Update

I posted this on Software Engineering and was told that it might be better here and on AI. I'm currently reading Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky (3rd ...
2
votes
0answers
107 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
0
votes
0answers
29 views

2-layered neural network with linear and sigmoid

Has a 2-layered neural network where each perceptron is a linear unit (no thresholding) has the same expresiveness as a 2-layered neural network where each perceptron is a sigmoid unit?
0
votes
2answers
36 views

No error in the training/validation dataset

Suppose I have a linearly separable dataset, divide into training and validation sets. Will a perceptron learned on the training dataset be guaranteed to have no error on the training dataset and on ...
0
votes
0answers
7 views

Why goal of PLA can ignore the norm of normal vector

Define hyperplane $w*x+b=0$, the goal of PLA(Perceptron Learning Algorithm) is minimizing the distance of misclassified points to the decision boundary, i.e. $$-\frac{1}{||w||}\sum_{i\in M} y_i(w*x_i ...
1
vote
1answer
177 views

Derivation of Perceptron weight update formula

I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
0
votes
1answer
31 views

Neural Networks - Back Propagation and Perceptrons

While studying about neural networks (still on basics - not Deep Learning etc.,) two questions came on my mind. What is the reason for replacing the hard limiter function in the nodes of the ...
0
votes
1answer
23 views

Is Perceptron a data-structure or an algorithm or both?

I found Perceptron in the discussions of (overlapping of disciplines is acknowledged): Neural Network Machine Learning Data Mining, Pattern Recognition Genetic Algorithm In the discussion of Neural ...
0
votes
0answers
15 views

Adaboost - Using Perceptrons

This is for a Image orientation project. I'm trying to implement adaboost with four perceptrons. The training data has four labels 0, 90, 180, 270. Each of the perceptron identifies one of the labels ...
0
votes
0answers
63 views

Training a perceptron on MNIST using whole images yielding perfect results

So I programmed a simple Perceptron algorithm to classify images from the MNIST dataset. My goal is to tell apart what image is a zero and what image is a one. First I trained my algorithm by ...
0
votes
1answer
16 views

Weight of MLP is larger than 1

I noticed when training MLP that weights of neurons can be larger than 1. Would this have negative effects on the outcome of the network? If yes, how to mitigate this problem?
0
votes
1answer
27 views

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality. Less cryptically put, MLP's are great ...
1
vote
0answers
38 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
102 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
0
votes
1answer
46 views

What is the expression for derivative of the signum function one should use in the BP training method

The back propagation learning method requires knowing of derivatives of activation functions. But what expression one should use for signum activation function $$ \mathrm{Signum}( x ) = \...
0
votes
1answer
45 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
0
votes
1answer
203 views

Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...
1
vote
1answer
564 views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
0
votes
0answers
183 views

What is the two class Bayes point machine?

I am trying to learn about the Bayes point machine, and more specifically the two class version. My googling returns only abstract explanations which don't help understanding the concept and novelty ...
0
votes
1answer
52 views

Conceptual question on MLP error calculation

Consider a NN of 2 input neurons, one hidden layer of 4 neurons and one output node. The task is to predict the next sample given two input samples at a time. $m_1$ is the output of the hidden layer, $...
2
votes
0answers
69 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
3
votes
1answer
279 views

MLP - what did author have in mind?

I am currently going through Machine learning and pattern recognition and I have following dilemma: A algorithm for MLP is presented in that way and corresponding python code to equation 4.9 (which ...
1
vote
1answer
742 views

How can a perceptron be used for regression?

I know perceptron is a binary classifier which has a 0/1 output. But in one of my exercises for a Neural Network course, there is a question that asks to implement a linear regression with perceptron. ...
2
votes
0answers
338 views

Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
0
votes
1answer
86 views

Clarification on simple perceptron neural network

I understand the title (and the question itself) is a little generic, but I have some questions that I doubt I can find from google search or studying the topic (yes, I've tried for a while). ...
1
vote
2answers
347 views

Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
0
votes
1answer
99 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...
12
votes
3answers
9k views

Multi-layer perceptron vs deep neural network

This is a question of terminology. Sometimes I see people refer to deep neural networks as "multi-layered perceptrons", why is this? A perceptron, I was taught, is a single layer classifier (or ...
1
vote
1answer
70 views

Why do nodes in hidden layer produce different results?

Assuming a simple, fully connected Multilayer Perceptron network with one input layer, one hidden layer with multiple nodes and one output layer. In this case the nodes in hidden layer are ...
0
votes
1answer
326 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
1
vote
1answer
700 views

Univariate time series multi step ahead prediction using multi-layer-perceptron (MLP)

I have a univariate time series data. I want to do a multi-step prediction. I came across this question which explains time series one step prediction. but I am interested in multi-step ahead ...
1
vote
1answer
181 views

Intuition behind neural networks

I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks. I'm following the Caltech video which is excellent https://www.youtube.com/watch?v=Ih5Mr93E-...
1
vote
1answer
93 views

Would multilayer perceptrons be better than multiple regression?

I am using multiple regression to predict the future value of a time series from several other time series. Would doing this with multilayer perceptrons produce better results than multiple regression?...
0
votes
2answers
2k views

What is weights in perceptron

I am just diving into machine learning and started with learning artificial neural networks. So on learning about perceptron I stucked on wording "weights". Is it rate of how much input item matched? ...
0
votes
1answer
372 views

Why does the weight vector in a perceptron monotonically tend to the generously feasible region

In a course on Machine Learning, in the chapter about a Perceptron, there is this statement: If a generously feasible region exists, then the distance between the current weight vector and a weight ...
0
votes
1answer
131 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
3answers
521 views

Does the adjustment / learning of the weights in Perceptron algorithm depend on the learning rate?

For perceptron algorithm, the output and target values are either $0$ or $1$. Assume output is $y$ and target is $d$. From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can ...
2
votes
2answers
1k views

Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
3
votes
2answers
2k views

How to Score an MLP Classifier

I am self-teaching Machine Learning, so excuse me if this is a stupid question. I am trying to understand MLP Classifiers, but would like to know what the best way to create a score is. e.g. ...
2
votes
0answers
1k views

Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
1
vote
0answers
423 views

Implementing a Perceptron in R - where am I going wrong?

I am trying to build a perception in R. I have generated some test data that is clearly linearly separable. I have tried a variety of learning rates, but the classification rate just seems to bounce ...
0
votes
2answers
1k views

(Deep) Neural Networks/MLPs: Should I normalize/scale my input features when the units of the features are meaningful?

Until now, I always normalized or standardized my features individually before feeding them into a neural network. But at my current project I have features, which in huge parts have the same unit (US-...
4
votes
3answers
5k views

Can a perceptron with sigmoid activation function perform nonlinear classification?

Consider the perceptron as illustrated in the figure above. I know: If the activation function is linear, i.e. the first three cases, then the perceptron is equivalent to a linear classifier. ...
-1
votes
2answers
83 views

Simplifying a perceptron model

I have a single percepton with sigmoid activation $y = f[\sum_{i=0}^{n}(x_i \cdot w_i)]$, where $f$ is a sigmoidal activation function. Is there a way to drop some of the inputs (or set their weigths ...
0
votes
1answer
23 views

Are B&W images of digits linear seperable

Given an image, expressed in the vector $\vec{v} = (v_1, \dots, v_n)\in \{0,1\}^n$ The vector $\vec{v}$ can represent images of the numbers 0 to 9. How do I know if this is linear seperable? Given ...