Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

1
vote
1answer
40 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
1
vote
1answer
226 views

Over which set of elements should I perform norm clipping of gradients for backpropagation?

I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the ...
1
vote
1answer
634 views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
1
vote
1answer
181 views

Intuition behind neural networks

I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks. I'm following the Caltech video which is excellent https://www.youtube.com/watch?v=Ih5Mr93E-...
0
votes
1answer
15 views

Training Perceptrons with Backprop

Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule? is it ...
0
votes
1answer
28 views

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality. Less cryptically put, MLP's are great ...
0
votes
1answer
48 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
0
votes
1answer
114 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...
0
votes
1answer
432 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
0
votes
1answer
142 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
1answer
121 views

Layers in neural networks

I am trying to train a MLP using some training data in which the inputs are 32x32 matrices and the final output is a scalar. Case 1: I build a network in which the first two hidden layers have the ...
0
votes
1answer
35 views

How to define user efficiency time

I'm making a user study. Users had to perform the same task 10 times. I've recorded the completion times of each task. Now I need to calculate the efficiency time. As you can imagine the user, when ...
0
votes
1answer
53 views

Single artificial neuron easily extendable to neural network

I'm working on implementation of artificial neuron which be extended to neural net. I want do implementation by myself to fully understand how it works. I start with perceptron with threshold ...
4
votes
0answers
770 views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
2
votes
0answers
114 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
2
votes
0answers
74 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
2
votes
0answers
356 views

Adding regularization term to perceptron weight update

I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the ...
2
votes
0answers
1k views

Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
2
votes
0answers
159 views

Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
2
votes
0answers
412 views

Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm

Recall the perceptron algorithm: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$ ...
1
vote
0answers
38 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
112 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
1
vote
0answers
438 views

Implementing a Perceptron in R - where am I going wrong?

I am trying to build a perception in R. I have generated some test data that is clearly linearly separable. I have tried a variety of learning rates, but the classification rate just seems to bounce ...
1
vote
0answers
165 views

Energy based model of perceptron loss?

I was reading LeCun's tutorial paper on energy-based models of cost functions, and came upon this table. I don't understand how he got the equation for perceptron loss. Can someone clarify how that ...
1
vote
0answers
174 views

Would a concatenated CBOW architecture outperform SG on rare words?

Typically, the Skip-Gram (SG) architecture is argued to be superior to Continuous Bag-of-Words (CBOW) for rare words because the CBOW averages over the input words (Mikolov et al., 2013). That is, ...
1
vote
0answers
37 views

Using Multi Layer Perceptron to categorize skills to roles

I have a data set of different roles and their skills. For eg. ...
1
vote
0answers
165 views

How does regularized logistic regression regularize perceptron hypothesis set in binary classifcation task?

I'm a newbie to Machine Learning and I'm not very good at math. I have read Learning From Data - A Short Course and met this Exercise 4.6 on page 133: We have seen both the hard-order constraint ...
1
vote
0answers
191 views

What is the standard format of training data for a MLP when the dataset includes various kind of data?

I am using a dataset that includes different kind of data: Real valued variables Binomial categorial variables Multilabel categorical variables I would like to build a deep multi-layer perceptron ...
1
vote
0answers
157 views

How to combine perceptron weights and probabilities

I am working on machine transliteration using structured perceptron to learn the model parameters at word level. I am using beam-search to extract the top $n$ transliterations of a word and would like ...
1
vote
0answers
349 views

Why use a restricted Boltzmann machine rather than a multi-layer perceptron?

I'm trying to understand the difference between a restricted Boltzmann machine (RBM), and a feed-forward neural network (NN). I know that an RBM is a generative model, where the idea is to reconstruct ...
1
vote
0answers
240 views

Perceptron Learning Algorithm: what is the probability that the viewed data is linearly separable, after some number of steps?

My understanding is that the Perceptron Learning Algorithm: will not converge if the data is not linearly separable. might take exponentially many iterations, even if the data is linearly separable. ...
0
votes
0answers
29 views

Z score standardisation vs min-max scaling for feature selection

I am applying l1 norm on the input weights of a single layer MLP. I wanted to know if I should standardize or min-max scale ([0 1] feature scaling) my input data?
0
votes
0answers
21 views

Choosing perceptron weights to achieve 0% error

I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
0
votes
0answers
30 views

2-layered neural network with linear and sigmoid

Has a 2-layered neural network where each perceptron is a linear unit (no thresholding) has the same expresiveness as a 2-layered neural network where each perceptron is a sigmoid unit?
0
votes
0answers
8 views

Why goal of PLA can ignore the norm of normal vector

Define hyperplane $w*x+b=0$, the goal of PLA(Perceptron Learning Algorithm) is minimizing the distance of misclassified points to the decision boundary, i.e. $$-\frac{1}{||w||}\sum_{i\in M} y_i(w*x_i ...
0
votes
0answers
15 views

Adaboost - Using Perceptrons

This is for a Image orientation project. I'm trying to implement adaboost with four perceptrons. The training data has four labels 0, 90, 180, 270. Each of the perceptron identifies one of the labels ...
0
votes
0answers
80 views

Training a perceptron on MNIST using whole images yielding perfect results

So I programmed a simple Perceptron algorithm to classify images from the MNIST dataset. My goal is to tell apart what image is a zero and what image is a one. First I trained my algorithm by ...