Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

Filter by
Sorted by
Tagged with
1
vote
1answer
2k views

Derivation of Perceptron weight update formula

I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
0
votes
1answer
40 views

Neural Networks - Back Propagation and Perceptrons

While studying about neural networks (still on basics - not Deep Learning etc.,) two questions came on my mind. What is the reason for replacing the hard limiter function in the nodes of the ...
0
votes
1answer
35 views

Is Perceptron a data-structure or an algorithm or both?

I found Perceptron in the discussions of (overlapping of disciplines is acknowledged): Neural Network Machine Learning Data Mining, Pattern Recognition Genetic Algorithm In the discussion of Neural ...
0
votes
1answer
72 views

Weight of MLP is larger than 1

I noticed when training MLP that weights of neurons can be larger than 1. Would this have negative effects on the outcome of the network? If yes, how to mitigate this problem?
0
votes
1answer
44 views

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality. Less cryptically put, MLP's are great ...
1
vote
0answers
39 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
235 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
1
vote
1answer
265 views

What is the expression for derivative of the signum function one should use in the BP training method

The back propagation learning method requires knowing of derivatives of activation functions. But what expression one should use for signum activation function $$ \mathrm{Signum}( x ) = \...
0
votes
1answer
75 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
2
votes
1answer
520 views

Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...
1
vote
1answer
1k views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
0
votes
1answer
139 views

Conceptual question on MLP error calculation

Consider a NN of 2 input neurons, one hidden layer of 4 neurons and one output node. The task is to predict the next sample given two input samples at a time. $m_1$ is the output of the hidden layer, $...
2
votes
0answers
99 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
3
votes
1answer
303 views

MLP - what did author have in mind?

I am currently going through Machine learning and pattern recognition and I have following dilemma: A algorithm for MLP is presented in that way and corresponding python code to equation 4.9 (which ...
1
vote
1answer
2k views

How can a perceptron be used for regression?

I know perceptron is a binary classifier which has a 0/1 output. But in one of my exercises for a Neural Network course, there is a question that asks to implement a linear regression with perceptron. ...
2
votes
0answers
592 views

Adding regularization term to perceptron weight update

I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the regularization term is shown in the ...
0
votes
1answer
163 views

Clarification on simple perceptron neural network

I understand the title (and the question itself) is a little generic, but I have some questions that I doubt I can find from google search or studying the topic (yes, I've tried for a while). ...
1
vote
2answers
506 views

Combining perceptrons

Has anyone ever tried to build a Multi-Layer Perceptron Neural Network without the sigmoid function? Let me explain better: We know that a perceptron is a binary classifier that assign the test ...
1
vote
2answers
203 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...
22
votes
3answers
18k views

Multi-layer perceptron vs deep neural network

This is a question of terminology. Sometimes I see people refer to deep neural networks as "multi-layered perceptrons", why is this? A perceptron, I was taught, is a single layer classifier (or ...
1
vote
1answer
206 views

Why do nodes in hidden layer produce different results?

Assuming a simple, fully connected Multilayer Perceptron network with one input layer, one hidden layer with multiple nodes and one output layer. In this case the nodes in hidden layer are ...
0
votes
1answer
751 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
2
votes
1answer
1k views

Univariate time series multi step ahead prediction using multi-layer-perceptron (MLP)

I have a univariate time series data. I want to do a multi-step prediction. I came across this question which explains time series one step prediction. but I am interested in multi-step ahead ...
1
vote
1answer
195 views

Intuition behind neural networks

I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks. I'm following the Caltech video which is excellent https://www.youtube.com/watch?v=Ih5Mr93E-...
1
vote
1answer
131 views

Would multilayer perceptrons be better than multiple regression?

I am using multiple regression to predict the future value of a time series from several other time series. Would doing this with multilayer perceptrons produce better results than multiple regression?...
0
votes
2answers
4k views

What is weights in perceptron

I am just diving into machine learning and started with learning artificial neural networks. So on learning about perceptron I stucked on wording "weights". Is it rate of how much input item matched? ...
0
votes
1answer
512 views

Why does the weight vector in a perceptron monotonically tend to the generously feasible region

In a course on Machine Learning, in the chapter about a Perceptron, there is this statement: If a generously feasible region exists, then the distance between the current weight vector and a weight ...
0
votes
1answer
214 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
3answers
745 views

Does the adjustment / learning of the weights in Perceptron algorithm depend on the learning rate?

For perceptron algorithm, the output and target values are either $0$ or $1$. Assume output is $y$ and target is $d$. From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can ...
3
votes
2answers
1k views

Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
2
votes
2answers
3k views

How to Score an MLP Classifier

I am trying to understand MLP Classifiers, but would like to know what the best way to create a score is. e.g. preferably a normalized score between 0 and 1. For instance, I looked at Scikit-learn's ...
3
votes
1answer
2k views

Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
1
vote
0answers
580 views

Implementing a Perceptron in R - where am I going wrong?

I am trying to build a perception in R. I have generated some test data that is clearly linearly separable. I have tried a variety of learning rates, but the classification rate just seems to bounce ...
0
votes
2answers
2k views

(Deep) Neural Networks/MLPs: Should I normalize/scale my input features when the units of the features are meaningful?

Until now, I always normalized or standardized my features individually before feeding them into a neural network. But at my current project I have features, which in huge parts have the same unit (US-...
6
votes
4answers
8k views

Can a perceptron with sigmoid activation function perform nonlinear classification?

Consider the perceptron as illustrated in the figure above. I know: If the activation function is linear, i.e. the first three cases, then the perceptron is equivalent to a linear classifier. ...
-1
votes
2answers
89 views

Simplifying a perceptron model

I have a single percepton with sigmoid activation $y = f[\sum_{i=0}^{n}(x_i \cdot w_i)]$, where $f$ is a sigmoidal activation function. Is there a way to drop some of the inputs (or set their weigths ...
0
votes
1answer
28 views

Are B&W images of digits linear seperable

Given an image, expressed in the vector $\vec{v} = (v_1, \dots, v_n)\in \{0,1\}^n$ The vector $\vec{v}$ can represent images of the numbers 0 to 9. How do I know if this is linear seperable? Given ...
1
vote
0answers
196 views

Energy based model of perceptron loss?

I was reading LeCun's tutorial paper on energy-based models of cost functions, and came upon this table. I don't understand how he got the equation for perceptron loss. Can someone clarify how that ...
1
vote
0answers
222 views

Would a concatenated CBOW architecture outperform SG on rare words?

Typically, the Skip-Gram (SG) architecture is argued to be superior to Continuous Bag-of-Words (CBOW) for rare words because the CBOW averages over the input words (Mikolov et al., 2013). That is, ...
1
vote
1answer
2k views

Question regarding weight update rule in Perceptron

In the single-layer perceptron, the weight update rule is given by:- $w_j := w_j + (y^{(i)} - \hat{y^{(i)}}) \times x_j^{(i)}$, where $w_j$ is the weight of the $j$th feature, $y^{(i)}$ is the ...
0
votes
1answer
980 views

Initialization of perceptron weights

In this deck about the perceptron, there's a pseudo-code for the batch version on page 12. During each iteration of the inner loop, the dot product $y_ix_i\theta$ is compared to 0, however since the ...
3
votes
1answer
1k views

Deriving the line for the decision boundary

Let's say I have the following perceptron activation line: $y = 1$ if $\sum_{i}^{d} w_ix_i \geq \theta $ and $y=0$ else. Now, for d = 4, I have derived the following question. $$ w_4x_4+w_3x_3+w_2x_2+...
1
vote
1answer
144 views

Combine two single-level perceptrons

Suppose we have a training dataset with $s$ instances, and we split it to 2 sets, of lengths $n, m$ such that $s = n + m$ . We train two perceptron models $p$ and $q$, both are defined by a k-...
1
vote
0answers
40 views

Using Multi Layer Perceptron to categorize skills to roles

I have a data set of different roles and their skills. For eg. ...
0
votes
1answer
165 views

neural network - target data

For the simple AND learning with a perceptron, it is required to have two inputs x1 and x2 and one target data y. Most AND examples (such as in the book "Fundamentals of neural network-fausett") have ...
1
vote
0answers
211 views

How does regularized logistic regression regularize perceptron hypothesis set in binary classifcation task?

I'm a newbie to Machine Learning and I'm not very good at math. I have read Learning From Data - A Short Course and met this Exercise 4.6 on page 133: We have seen both the hard-order constraint ...
1
vote
0answers
195 views

What is the standard format of training data for a MLP when the dataset includes various kind of data?

I am using a dataset that includes different kind of data: Real valued variables Binomial categorial variables Multilabel categorical variables I would like to build a deep multi-layer perceptron ...
5
votes
2answers
12k views

What is the difference between MLP and RBF?

What are the main differences between two types of feedforward networks such as multilayer perceptrons (MLP) and radial basis function (RBF)? What are the fundamental differences between these two ...
0
votes
1answer
138 views

Layers in neural networks

I am trying to train a MLP using some training data in which the inputs are 32x32 matrices and the final output is a scalar. Case 1: I build a network in which the first two hidden layers have the ...
2
votes
0answers
69 views

Perceptron trained on time series always predicting the same answer [duplicate]

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...