Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Why in gated recurrent unit gates are controlled by only one layer perceptrons?

Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
2
votes
1answer
32 views

Whats the different between Logistic regression and perceptron?

In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in classification results, since they will have the same decision ...
0
votes
0answers
7 views

Calculate the confidence score (decision_function) of perceptron, by the signed distance of that sample to the hyperplane

I've implemented the binary version of perceptron from scratch, in python. I would like to use it for one vs all classification, by using the one vs all of sklearn. for that, I need to implement the ...
1
vote
1answer
26 views

Can we express CNNs in terms of a MLP?

I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
3
votes
0answers
86 views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
0
votes
1answer
15 views

Neural network regression with constraint

I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall ...
2
votes
1answer
56 views

Can a 4D perceptron be plotted in 2 dimensions?

I am wondering if it is at all possible to plot a 4D perceptron line in 2D. Obviously, it would be impossible to observe it with all of its original information, but is there a way for me to observe ...
1
vote
1answer
20 views

Can we use perceptron training algorithm to train a single neuron with sigmoid activation?

The perceptron training algorithm is summarized as: Apply the inputs and calculate the output $ y $ Compare with the desired output yd and calculate error $e = y-y_d$ Update the weights based on the ...
5
votes
2answers
67 views

SciKit Learn: Multilayer perceptron early stopping, restore best weights

In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
0
votes
0answers
14 views

Weight update rule for Rosenblatt's Perceptron

I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron: $w_{i, j} = w_{i, j} +\eta ( y_j - \hat{y}_j ) x_j$ It seems to me that the step ...
0
votes
0answers
28 views

Use a multilevel logistic regression and cross validation

I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
0
votes
1answer
24 views

Am I fundamentally misunderstanding the net input dot product w*x

Most books have the notation of a weight vector w and input matrix x: $$ w = \begin{bmatrix} w_1\\...\\ w_D \end{bmatrix}, x = \begin{bmatrix} x_{11}&...&x_{1D}\\ ...&...&...\\ x_{N1}&...
1
vote
1answer
27 views

Equivalence of Deep Feed-Forward Neural Network to Single-Layer Network

I am new to Deep Learning, and I was going through some lecture notes I found online. It said that a feed-forward neural network with $n$ hidden layers and only linear activation functions is ...
2
votes
0answers
11 views

Can we get the input from a multilayer perceptron based on the output of one of its hidden layers?

I was reading a relatively new paper that proposed to split a nerual networks layers into groups and sending each group to different nodes to train them in a distributed manner. In order to not send ...
2
votes
1answer
71 views

Can non-linearly separable data always be made linearly separable?

A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
1
vote
1answer
48 views

Kernelized perceptron algorithm weights update

I'm asked to find the maximum margin decision surface separating positive from negative samples by inspection. The positive examples are (1,1) and (-1,-1), the negative ones are (1,-1) and (-1,1). The ...
2
votes
1answer
41 views

Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
1
vote
2answers
38 views

LabelEncoder with a Multi-Layer Perceptron?

So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline: ...
0
votes
0answers
20 views

SSE over epochs for MLPClassifier

I have a simple csv dataset I want to do something simple. Use the multilayer perceptron algorithm and plot SSE over epochs. I am a novice, I have searched a lot but cannot find a good solution. How ...
0
votes
2answers
114 views

Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?

I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. ...
0
votes
0answers
61 views

Working of Dual Perceptron Algorithm

I was going for the theory and maths behind the online perceptron algorithm and it is very easy to under stand it intuitively that on a positive mistake, you just add the ...
7
votes
1answer
176 views

Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss setup ...
0
votes
0answers
32 views

Why does the decision boundary of a perceptron shift when all weights are multiplied by the same factor?

Visit the TensorFlow Playground at https://playground.tensorflow.org/. Use the following configuration (leave all settings at default except the following): Choose "Gaussian" dataset Set activation ...
0
votes
0answers
30 views

Perceptron results in WEKA. Which are the weights & how to graph sigmoid function?

i have this data set dataset.csv and these results ...
0
votes
0answers
41 views

Multilayer Perceptron with XOR Dataset

so I got this working code for a multilayer perceptron class, which I'm training on a XOR dataset (plot 1). As activation I'm using the hyperbolic tangent. After 50000 training epochs using SGD, my ...
1
vote
1answer
27 views

Why do my training losses go up?

I am new to Machine Learning and Tensorflow. For one of my courses, I need to train an MLP for the xor gate. But my losses somehow go up each epoch, which confuses me and I must admit that I ran out ...
0
votes
0answers
19 views

Why does the MLP not work? [duplicate]

I feel like I implemented it right, but somehow my accuracy stays 50%. I just wanna know what kind of mistake it is. https://colab.research.google.com/drive/1KlnqmunviSkKaEiT5PJ98zsIzK_7Zrcv#scrollTo=...
0
votes
0answers
25 views

How to choose the number of hidden units in MLP classifier?

I have a huge datset (65.000 instances, 13 features). I know empirical rules regarding to the choice of number of units in the hidden layer (e.g. units <= 2*#features), but I obtained better ...
0
votes
0answers
11 views

Is a multilayer perceptron feasible/advisable when the number of samples for each class can be expected to be 100 or fewer?

I am a beginner, and trying to understand which parameters to choose for a machine learning task I'd like to solve with a multilayer perceptron/NN. I believe it compares to MNIST in a way, but has ...
0
votes
0answers
41 views

Kernelized Perceptron, is it a more efficient algorithm?

I'm reading http://ciml.info/ chapter 11 Kernel Methods. For a feature vector x=x1,x2,x3,...,xD, feature combination expands O(x^2) features. We can rewrite linear models which only takes O(x) ...
0
votes
1answer
91 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
1
vote
1answer
75 views

Can a unit-step function introduce non-linearity if used in the hidden nodes?

I have a follow up from this question. From the answers in that question, it's obvious that at minimum, a shallow network (only has a single hidden layer), as defined in the "Universal Approximation ...
0
votes
1answer
29 views

linear perceptron algorithm with 3 training samples

So I am working on a linear perceptron algorithm problem that has 3 training samples. (2D space) ...
0
votes
1answer
1k views

Gradient descent with Binary Cross-Entropy for single layer perceptron

I'm implementing a Single Layer Perceptron for binary classification in python. I'm using binary Cross-Entropy loss function and gradient descent. The gradient descent is not converging, may be I'm ...
0
votes
0answers
135 views

what is the cost function for a perceptron muticap

I have the function of cost or error of a perceptron of an entry and exit ...
1
vote
0answers
23 views

How are the weights of a single layer perceptron updated given the misclassification of a point?

Here is some data separated by decision line The equation of decision line/boundary is $x_1 -3x_2 + 3 = 0$. Therefore $w_1 = 1, w_2 = -3$ and, if $w_0 = -1$, then the bias $\theta = -3$. Now, I have ...
1
vote
1answer
120 views

Why does Perceptron use L1 norm as its error function?

Usually, people use L2 norm as a machine learning error function. per wiki the error function for a Perceptron model is ${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|}$ why is that?
0
votes
0answers
33 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
0
votes
0answers
125 views

How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
1
vote
1answer
62 views

Rule of thumb Overfit in a MLP or is it possible with N = 135

I want to say ahead that I highly appreciate any Literature recommendation (Book, blogg, ...) besides ESL and ISL. My Question: Is it possible to train a 3 layer multilayer perceptron (mlp) in a ...
1
vote
2answers
642 views

How are the weights updated in the perceptron learning rule?

I'm considering a perceptron model. I know that when feeding observations from the training dataset to the model, if the model correctly classifies the input, then the weights for this input will not ...
2
votes
2answers
584 views

Why perceptron is linear classifier?

It is said that perceptron is linear classifier, but it has a non-linear activation function f = 1 if wx - b >= 0 and f = 0 otherwise If i will use some non-linear function on linear combination of ...
1
vote
1answer
25 views

Training Perceptrons with Backprop

Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule? is it ...
1
vote
1answer
72 views

No need for bias term if data is standardised? Linear classification models

For linear classification models, e.g. perceptron, bias term allows to move separating hyperplane away from origin. If data is scattered around the zero does that mean that we don't need bias term?
0
votes
0answers
25 views

Choosing perceptron weights to achieve 0% error

I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
1
vote
1answer
942 views

Perceptron learning for non-linearly separable data

It is well known that perceptron learning will never converge for non-linearly separable data. This means that you cannot fit a hyperplane in any dimensions that would separate the two classes. Is it ...
1
vote
1answer
54 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
0
votes
1answer
77 views

Neural Network Weight Update

I posted this on Software Engineering and was told that it might be better here and on AI. I'm currently reading Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky (3rd ...
1
vote
1answer
214 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
0
votes
2answers
302 views

No error in the training/validation dataset

Suppose I have a linearly separable dataset, divide into training and validation sets. Will a perceptron learned on the training dataset be guaranteed to have no error on the training dataset and on ...