Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

0
votes
0answers
22 views

2-layered neural network with linear and sigmoid

Has a 2-layered neural network where each perceptron is a linear unit (no thresholding) has the same expresiveness as a 2-layered neural network where each perceptron is a sigmoid unit?
0
votes
2answers
32 views

No error in the training/validation dataset

Suppose I have a linearly separable dataset, divide into training and validation sets. Will a perceptron learned on the training dataset be guaranteed to have no error on the training dataset and on ...
0
votes
0answers
5 views

Why goal of PLA can ignore the norm of normal vector

Define hyperplane $w*x+b=0$, the goal of PLA(Perceptron Learning Algorithm) is minimizing the distance of misclassified points to the decision boundary, i.e. $$-\frac{1}{||w||}\sum_{i\in M} y_i(w*x_i ...
1
vote
1answer
60 views

Derivation of Perceptron weight update formula

I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
0
votes
1answer
27 views

Neural Networks - Back Propagation and Perceptrons

While studying about neural networks (still on basics - not Deep Learning etc.,) two questions came on my mind. What is the reason for replacing the hard limiter function in the nodes of the ...
0
votes
1answer
21 views

Is Perceptron a data-structure or an algorithm or both?

I found Perceptron in the discussions of (overlapping of disciplines is acknowledged): Neural Network Machine Learning Data Mining, Pattern Recognition Genetic Algorithm In the discussion of Neural ...
0
votes
0answers
11 views

Adaboost - Using Perceptrons

This is for a Image orientation project. I'm trying to implement adaboost with four perceptrons. The training data has four labels 0, 90, 180, 270. Each of the perceptron identifies one of the labels ...
0
votes
0answers
26 views

Training a perceptron on MNIST using whole images yielding perfect results

So I programmed a simple Perceptron algorithm to classify images from the MNIST dataset. My goal is to tell apart what image is a zero and what image is a one. First I trained my algorithm by ...
0
votes
1answer
9 views

Weight of MLP is larger than 1

I noticed when training MLP that weights of neurons can be larger than 1. Would this have negative effects on the outcome of the network? If yes, how to mitigate this problem?
0
votes
1answer
23 views

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality. Less cryptically put, MLP's are great ...
1
vote
0answers
37 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
73 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
0
votes
1answer
27 views

What is the expression for derivative of the signum function one should use in the BP training method

The back propagation learning method requires knowing of derivatives of activation functions. But what expression one should use for signum activation function $$ \mathrm{Signum}( x ) = \...
0
votes
1answer
37 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
0
votes
1answer
125 views

Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...
1
vote
1answer
454 views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
0
votes
0answers
151 views

What is the two class Bayes point machine?

I am trying to learn about the Bayes point machine, and more specifically the two class version. My googling returns only abstract explanations which don't help understanding the concept and novelty ...
0
votes
1answer
48 views

Conceptual question on MLP error calculation

Consider a NN of 2 input neurons, one hidden layer of 4 neurons and one output node. The task is to predict the next sample given two input samples at a time. $m_1$ is the output of the hidden layer, $...
2
votes
0answers
67 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
0
votes
0answers
213 views

How to improve accuracy my multi-layer perceptron model?

I am currently training a multi-layer perceptron model on 1200 real values features with binary class labels (0 and 1) and my optimizer is SGD. I have tried to tune with respect to number of layers of ...
3
votes
1answer
274 views

MLP - what did author have in mind?

I am currently going through Machine learning and pattern recognition and I have following dilemma: A algorithm for MLP is presented in that way and corresponding python code to equation 4.9 (which ...
1
vote
1answer
607 views

How can a perceptron be used for regression?

I know perceptron is a binary classifier which has a 0/1 output. But in one of my exercises for a Neural Network course, there is a question that asks to implement a linear regression with perceptron. ...
0
votes
0answers
103 views

number of neurons for mnist dataset using mlp?

I am trying to find out what is optimum number of neurons that can be used in MNIST dataset(60,000 training and 10,000 testing data). I build a single hidden layer model using keras,with relu ...
0
votes
0answers
79 views

Perceptron algorithm: permuting training data every iteration

Permuting the training data each iteration saves somewhere between 20% and 50% of your time, are there any cases in which you might not want to permute the data every iteration? Source :http://www....
2
votes
0answers
246 views

Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
0
votes
1answer
80 views

Clarification on simple perceptron neural network

I understand the title (and the question itself) is a little generic, but I have some questions that I doubt I can find from google search or studying the topic (yes, I've tried for a while). ...
1
vote
2answers
313 views

Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
0
votes
1answer
66 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...
9
votes
2answers
8k views

Multi-layer perceptron vs deep neural network

This is a question of terminology. Sometimes I see people refer to deep neural networks as "multi-layered perceptrons", why is this? A perceptron, I was taught, is a single layer classifier (or ...
1
vote
1answer
67 views

Why do nodes in hidden layer produce different results?

Assuming a simple, fully connected Multilayer Perceptron network with one input layer, one hidden layer with multiple nodes and one output layer. In this case the nodes in hidden layer are ...
0
votes
1answer
266 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
1
vote
1answer
653 views

Univariate time series multi step ahead prediction using multi-layer-perceptron (MLP)

I have a univariate time series data. I want to do a multi-step prediction. I came across this question which explains time series one step prediction. but I am interested in multi-step ahead ...
1
vote
1answer
168 views

Intuition behind neural networks

I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks. I'm following the Caltech video which is excellent https://www.youtube.com/watch?v=Ih5Mr93E-...
1
vote
1answer
87 views

Would multilayer perceptrons be better than multiple regression?

I am using multiple regression to predict the future value of a time series from several other time series. Would doing this with multilayer perceptrons produce better results than multiple regression?...
0
votes
2answers
2k views

What is weights in perceptron

I am just diving into machine learning and started with learning artificial neural networks. So on learning about perceptron I stucked on wording "weights". Is it rate of how much input item matched? ...
0
votes
1answer
329 views

Why does the weight vector in a perceptron monotonically tend to the generously feasible region

In a course on Machine Learning, in the chapter about a Perceptron, there is this statement: If a generously feasible region exists, then the distance between the current weight vector and a weight ...
0
votes
1answer
128 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
3answers
439 views

Does the adjustment / learning of the weights in Perceptron algorithm depend on the learning rate?

For perceptron algorithm, the output and target values are either $0$ or $1$. Assume output is $y$ and target is $d$. From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can ...
2
votes
2answers
939 views

Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
3
votes
2answers
2k views

How to Score an MLP Classifier

I am self-teaching Machine Learning, so excuse me if this is a stupid question. I am trying to understand MLP Classifiers, but would like to know what the best way to create a score is. e.g. ...
2
votes
0answers
1k views

Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
1
vote
0answers
374 views

Implementing a Perceptron in R - where am I going wrong?

I am trying to build a perception in R. I have generated some test data that is clearly linearly separable. I have tried a variety of learning rates, but the classification rate just seems to bounce ...
0
votes
2answers
1k views

(Deep) Neural Networks/MLPs: Should I normalize/scale my input features when the units of the features are meaningful?

Until now, I always normalized or standardized my features individually before feeding them into a neural network. But at my current project I have features, which in huge parts have the same unit (US-...
4
votes
3answers
5k views

Can a perceptron with sigmoid activation function perform nonlinear classification?

Consider the perceptron as illustrated in the figure above. I know: If the activation function is linear, i.e. the first three cases, then the perceptron is equivalent to a linear classifier. ...
-1
votes
2answers
80 views

Simplifying a perceptron model

I have a single percepton with sigmoid activation $y = f[\sum_{i=0}^{n}(x_i \cdot w_i)]$, where $f$ is a sigmoidal activation function. Is there a way to drop some of the inputs (or set their weigths ...
0
votes
1answer
23 views

Are B&W images of digits linear seperable

Given an image, expressed in the vector $\vec{v} = (v_1, \dots, v_n)\in \{0,1\}^n$ The vector $\vec{v}$ can represent images of the numbers 0 to 9. How do I know if this is linear seperable? Given ...
1
vote
0answers
147 views

Energy based model of perceptron loss?

I was reading LeCun's tutorial paper on energy-based models of cost functions, and came upon this table. I don't understand how he got the equation for perceptron loss. Can someone clarify how that ...
1
vote
0answers
139 views

Would a concatenated CBOW architecture outperform SG on rare words?

Typically, the Skip-Gram (SG) architecture is argued to be superior to Continuous Bag-of-Words (CBOW) for rare words because the CBOW averages over the input words (Mikolov et al., 2013). That is, ...
1
vote
1answer
1k views

Question regarding weight update rule in Perceptron

In the single-layer perceptron, the weight update rule is given by:- $w_j := w_j + (y^{(i)} - \hat{y^{(i)}}) \times x_j^{(i)}$, where $w_j$ is the weight of the $j$th feature, $y^{(i)}$ is the ...
0
votes
1answer
713 views

Initialization of perceptron weights

In this deck about the perceptron, there's a pseudo-code for the batch version on page 12. During each iteration of the inner loop, the dot product $y_ix_i\theta$ is compared to 0, however since the ...