Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

49 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
1k views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
3
votes
0answers
87 views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
2
votes
1answer
44 views

Whats the different between Logistic regression and perceptron?

In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in classification results, since they will have the same decision ...
2
votes
0answers
12 views

Can we get the input from a multilayer perceptron based on the output of one of its hidden layers?

I was reading a relatively new paper that proposed to split a nerual networks layers into groups and sending each group to different nodes to train them in a distributed manner. In order to not send ...
2
votes
2answers
630 views

Why perceptron is linear classifier?

It is said that perceptron is linear classifier, but it has a non-linear activation function f = 1 if wx - b >= 0 and f = 0 otherwise If i will use some non-linear function on linear combination of ...
2
votes
0answers
99 views

Output node of Perceptron neural network 'learning' unexpected function

I'm building some simple Perceptron networks to gain insight into how they operate. Most of the results are compelling, but there is one that I cannot figure out. Here is my simple Perceptron class ...
2
votes
0answers
591 views

Adding regularization term to perceptron weight update

I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes... Usually the regularization term is shown in the ...
2
votes
0answers
165 views

Machine learning: intuition behind perceptron learning algorithm

Given features x_1...x_n, weights w_1...w_n, calculated output y = Z dot X, and actual output y', the perceptron learning algorithm changes the weights after each iteration as follows: ...
2
votes
0answers
669 views

Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm

Recall the perceptron algorithm: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$ ...
1
vote
1answer
28 views

Can we express CNNs in terms of a MLP?

I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
1
vote
1answer
20 views

Can we use perceptron training algorithm to train a single neuron with sigmoid activation?

The perceptron training algorithm is summarized as: Apply the inputs and calculate the output $ y $ Compare with the desired output yd and calculate error $e = y-y_d$ Update the weights based on the ...
1
vote
2answers
38 views

LabelEncoder with a Multi-Layer Perceptron?

So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline: ...
1
vote
1answer
27 views

Why do my training losses go up?

I am new to Machine Learning and Tensorflow. For one of my courses, I need to train an MLP for the xor gate. But my losses somehow go up each epoch, which confuses me and I must admit that I ran out ...
1
vote
0answers
25 views

How are the weights of a single layer perceptron updated given the misclassification of a point?

Here is some data separated by decision line The equation of decision line/boundary is $x_1 -3x_2 + 3 = 0$. Therefore $w_1 = 1, w_2 = -3$ and, if $w_0 = -1$, then the bias $\theta = -3$. Now, I have ...
1
vote
1answer
54 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
1
vote
1answer
219 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
1
vote
0answers
39 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
233 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
1
vote
2answers
202 views

How to find multi-layer perceptron weights?

I want to use a multi-layer perceptron to design the following function : The architecture I want to use is the following one : What would be $w_i$ weights ? Is there any guide to find them ? I ...
1
vote
0answers
579 views

Implementing a Perceptron in R - where am I going wrong?

I am trying to build a perception in R. I have generated some test data that is clearly linearly separable. I have tried a variety of learning rates, but the classification rate just seems to bounce ...
1
vote
0answers
196 views

Energy based model of perceptron loss?

I was reading LeCun's tutorial paper on energy-based models of cost functions, and came upon this table. I don't understand how he got the equation for perceptron loss. Can someone clarify how that ...
1
vote
0answers
221 views

Would a concatenated CBOW architecture outperform SG on rare words?

Typically, the Skip-Gram (SG) architecture is argued to be superior to Continuous Bag-of-Words (CBOW) for rare words because the CBOW averages over the input words (Mikolov et al., 2013). That is, ...
1
vote
0answers
40 views

Using Multi Layer Perceptron to categorize skills to roles

I have a data set of different roles and their skills. For eg. ...
1
vote
0answers
211 views

How does regularized logistic regression regularize perceptron hypothesis set in binary classifcation task?

I'm a newbie to Machine Learning and I'm not very good at math. I have read Learning From Data - A Short Course and met this Exercise 4.6 on page 133: We have seen both the hard-order constraint ...
1
vote
0answers
195 views

What is the standard format of training data for a MLP when the dataset includes various kind of data?

I am using a dataset that includes different kind of data: Real valued variables Binomial categorial variables Multilabel categorical variables I would like to build a deep multi-layer perceptron ...
1
vote
0answers
167 views

How to combine perceptron weights and probabilities

I am working on machine transliteration using structured perceptron to learn the model parameters at word level. I am using beam-search to extract the top $n$ transliterations of a word and would like ...
1
vote
0answers
401 views

Why use a restricted Boltzmann machine rather than a multi-layer perceptron?

I'm trying to understand the difference between a restricted Boltzmann machine (RBM), and a feed-forward neural network (NN). I know that an RBM is a generative model, where the idea is to reconstruct ...
1
vote
0answers
280 views

Perceptron Learning Algorithm: what is the probability that the viewed data is linearly separable, after some number of steps?

My understanding is that the Perceptron Learning Algorithm: will not converge if the data is not linearly separable. might take exponentially many iterations, even if the data is linearly separable. ...
0
votes
0answers
16 views

Why in gated recurrent unit gates are controlled by only one layer perceptrons?

Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
0
votes
0answers
9 views

Calculate the confidence score (decision_function) of perceptron, by the signed distance of that sample to the hyperplane

I've implemented the binary version of perceptron from scratch, in python. I would like to use it for one vs all classification, by using the one vs all of sklearn. for that, I need to implement the ...
0
votes
0answers
14 views

Weight update rule for Rosenblatt's Perceptron

I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron: $w_{i, j} = w_{i, j} +\eta ( y_j - \hat{y}_j ) x_j$ It seems to me that the step ...
0
votes
0answers
29 views

Use a multilevel logistic regression and cross validation

I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
0
votes
0answers
24 views

SSE over epochs for MLPClassifier

I have a simple csv dataset I want to do something simple. Use the multilayer perceptron algorithm and plot SSE over epochs. I am a novice, I have searched a lot but cannot find a good solution. How ...
0
votes
0answers
65 views

Working of Dual Perceptron Algorithm

I was going for the theory and maths behind the online perceptron algorithm and it is very easy to under stand it intuitively that on a positive mistake, you just add the ...
0
votes
0answers
34 views

Why does the decision boundary of a perceptron shift when all weights are multiplied by the same factor?

Visit the TensorFlow Playground at https://playground.tensorflow.org/. Use the following configuration (leave all settings at default except the following): Choose "Gaussian" dataset Set activation ...
0
votes
0answers
31 views

Perceptron results in WEKA. Which are the weights & how to graph sigmoid function?

i have this data set dataset.csv and these results ...
0
votes
0answers
43 views

Multilayer Perceptron with XOR Dataset

so I got this working code for a multilayer perceptron class, which I'm training on a XOR dataset (plot 1). As activation I'm using the hyperbolic tangent. After 50000 training epochs using SGD, my ...
0
votes
0answers
27 views

How to choose the number of hidden units in MLP classifier?

I have a huge datset (65.000 instances, 13 features). I know empirical rules regarding to the choice of number of units in the hidden layer (e.g. units <= 2*#features), but I obtained better ...
0
votes
0answers
11 views

Is a multilayer perceptron feasible/advisable when the number of samples for each class can be expected to be 100 or fewer?

I am a beginner, and trying to understand which parameters to choose for a machine learning task I'd like to solve with a multilayer perceptron/NN. I believe it compares to MNIST in a way, but has ...
0
votes
0answers
42 views

Kernelized Perceptron, is it a more efficient algorithm?

I'm reading http://ciml.info/ chapter 11 Kernel Methods. For a feature vector x=x1,x2,x3,...,xD, feature combination expands O(x^2) features. We can rewrite linear models which only takes O(x) ...
0
votes
1answer
30 views

linear perceptron algorithm with 3 training samples

So I am working on a linear perceptron algorithm problem that has 3 training samples. (2D space) ...
0
votes
0answers
137 views

what is the cost function for a perceptron muticap

I have the function of cost or error of a perceptron of an entry and exit ...
0
votes
0answers
25 views

Choosing perceptron weights to achieve 0% error

I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
0
votes
1answer
74 views

Perceptron Learning Algorithm

Is it possible to draw a hyperplane/decision boundary (not a decision function) from the parameters learned in PLA. If so how is this done (code would be ideal, if available). If not why?
0
votes
1answer
747 views

Update weight vector regardless of correctness for perceptron algorithm

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
0
votes
1answer
205 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
0
votes
1answer
35 views

How to define user efficiency time

I'm making a user study. Users had to perform the same task 10 times. I've recorded the completion times of each task. Now I need to calculate the efficiency time. As you can imagine the user, when ...
0
votes
1answer
56 views

Single artificial neuron easily extendable to neural network

I'm working on implementation of artificial neuron which be extended to neural net. I want do implementation by myself to fully understand how it works. I start with perceptron with threshold ...