Questions tagged [perceptron]

An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.

Filter by
Sorted by
Tagged with
0 votes
0 answers
32 views

Why is Binary data better than discrete numerical data for Perceptron?

I've heard that when using classifier-type machine learning algorithms (in my case perceptron) it's better to have all fields be binary than to have a mixture of binary and numerical fields. If ...
user avatar
  • 1
1 vote
0 answers
17 views

What are the differences between the perceptron and linear regression hypothesis set?

While studying machine learning I have known 2 learning models: linear regression and perceptron. I know the difference between the Learning algorithm they use, but the hypothesis set look the same to ...
user avatar
  • 41
1 vote
0 answers
12 views

Differentiating output of layer with respect to its input [duplicate]

Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$. I was ...
user avatar
  • 11
1 vote
1 answer
58 views

What is the interpretation of sklearn's linear perceptron coefficients?

I'm stumped as to why this example doesn't do a better job fitting the data, I suspect it has to do with my interpretation of the perceptron object's coefficients. Note that I'm interested in the <...
user avatar
0 votes
0 answers
40 views

Convergence theorems for Kernel Perceptron and Kernel SVM

Context Some time ago I asked whether SVMs could work on arbitrary Hilbert spaces, my motivation for asking it was due to my discomfort towards the kernelized version of SVM, which, in my mind, ...
user avatar
0 votes
1 answer
37 views

Different results for Logistic Regression (wrong) and Perceptron (correct)

To help me with some understanding, I'm trying to learn the Logical AND and Logical OR using Linear Regression trained over the following data: ...
user avatar
  • 125
0 votes
0 answers
22 views

Why do the training and validation accuracy drop after tuning?

I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped. Then someone told me that I should add Dropout layers with 50% dropping. I did that. ...
user avatar
  • 1,518
0 votes
1 answer
54 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
user avatar
  • 3
1 vote
1 answer
51 views

Single-layer perceptron mathematical formulation

I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then $$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\...
user avatar
0 votes
1 answer
69 views

How to show that the gradient of the smoothed surrogate loss function leads to perceptron update?

This is about the contents of section 1.2.1 and 1.2.1.1 of the book "Neural Networks and Deep Learning: A Textbook". The link to the sections is here. The question arises from the following ...
user avatar
  • 1
0 votes
0 answers
13 views

Radial Basis Network and Multilayer Perceptron Network regression in R

I am new to R and I try to make a simple regression problem using a radial basis neural network and a multilayer perceptron neural network. With the following code: ...
user avatar
0 votes
1 answer
42 views

Did Hinton introduce the concept of distributed representation?

From Goodfellow et al.'s Deep Learning book: Several key concepts arose during (...) the 1980s that remain central to today’s deep learning. One of these concepts is that of distributed ...
user avatar
0 votes
0 answers
12 views

Clarification needed for a proof step in the paper "Perceptron Mistake Bounds"

I was trying to understand the section 3.1 L1 norm mistake bound (for non-separable case). In the proof of theorem 2, there is a step that takes into account the property of convexity and derives an ...
user avatar
  • 1
1 vote
1 answer
43 views

Stop criterion is Infinitive in Perceptron in Sklearn

I read code in book "Hand-on Machine Learning in Sklearn and TensorFlow" by Aurelien Geron ...
user avatar
0 votes
0 answers
84 views

Volatility forecasting using MLP

I am currently working on a project which aims to predict the Monthly volatility of the S&P 500 index with the aid of Multilayer Perceptrons (MLP). Actually, I am trying to reproduce some of the ...
user avatar
  • 75
17 votes
1 answer
3k views

Why pure exponent is not used as activation function for neural networks?

The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU). The overall idea of these functions is the same: before ...
user avatar
0 votes
0 answers
85 views

When do Tanh activation functions outperform Relu activation functions?

I have a somewhat generic question: I have been researching about what activation function to use in the hidden layers of a given neural network. But it seems that in literature, the standard Relu ...
user avatar
2 votes
1 answer
150 views

How to resolve the perceptron dilemma for binary classification?

I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
user avatar
  • 674
0 votes
1 answer
19 views

In AI/ML, using the Perceptron model, would it ever make sense to have both negative weights and data?

I understand the math but I want to make sure I understand the mapping back to real world scenarios. Thinking about it logically, I cannot think of a real world scenario where you would have a ...
user avatar
0 votes
1 answer
412 views

In what way are SGD and Perceptron very similar?

I'm reading Hands-On Machine Learning and the author states that: You may have noticed the fact that the Perceptron learning algorithm strongly resembles Stochastic Gradient Descent. In fact, Scikit-...
user avatar
1 vote
0 answers
66 views

One-hot encoding in Keras for R [closed]

I am trying to build a binary classifier using a MLP with the Keras package in R. My question is, why does the package require the labels to be a one-hot vector? For example, the value 1 will be the ...
user avatar
3 votes
1 answer
62 views

Best approach for energy demand forecasting

I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this: ...
user avatar
  • 53
1 vote
1 answer
311 views

How to draw the single perceptron decision boundary when weights and bias are 0?

I've been following an algorithm described on a book called Knowledge Discovery with Support Vector Machines by Lutz H. Hamel. In the book, there is this learning algorithm for a single perceptron ...
user avatar
3 votes
1 answer
74 views

About Multi-layer Perceptrons

I've always been a bit confused when it comes to Deep Learning terminology. Is the definition of the perceptron, whether single layer or multi layer, associated with a specific type of activation ...
user avatar
1 vote
0 answers
54 views

Rule of thumb for Data Requirements when Designing a Neural Network for Deep Learning

I'm designing an MLP classifier and I've been noticing that: Using a very shallow network, or one whose at least one layer has a small number of neurons yields bad performance Using a deep network ...
user avatar
  • 111
1 vote
1 answer
182 views

Perceptron as a Logistic Regression

If by following way single perceptron is made to work like Logistic Regression. How much correct is it to say that I made perceptron to work as Logistic Regression. Question came to mind as ...
user avatar
  • 155
1 vote
0 answers
48 views

Why do we use Matrix in Perceptron instead of Functions?

Matrices are good objects to store connections between dimensions/entities. However, matrix computation is often time consuming and sometimes wasteful if matrix is too sparse. Also thinking about the ...
user avatar
  • 111
2 votes
0 answers
102 views

sklearn Perceptron incorrectly training on tiny 3 point linearly separable 2D dataset?

...
user avatar
0 votes
0 answers
25 views

Why in gated recurrent unit gates are controlled by only one layer perceptrons?

Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
user avatar
  • 1
2 votes
1 answer
527 views

Whats the different between Logistic regression and perceptron?

In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in classification results, since they will have the same decision ...
user avatar
  • 21
1 vote
1 answer
117 views

Can we express CNNs in terms of a MLP?

I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
user avatar
5 votes
1 answer
865 views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
user avatar
0 votes
1 answer
524 views

Neural network regression with constraint

I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall ...
user avatar
  • 1,580
2 votes
1 answer
407 views

Can a 4D perceptron be plotted in 2 dimensions?

I am wondering if it is at all possible to plot a 4D perceptron line in 2D. Obviously, it would be impossible to observe it with all of its original information, but is there a way for me to observe ...
user avatar
  • 23
2 votes
1 answer
153 views

Can we use perceptron training algorithm to train a single neuron with sigmoid activation?

The perceptron training algorithm is summarized as: Apply the inputs and calculate the output $ y $ Compare with the desired output yd and calculate error $e = y-y_d$ Update the weights based on the ...
user avatar
5 votes
2 answers
616 views

SciKit Learn: Multilayer perceptron early stopping, restore best weights

In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
user avatar
0 votes
1 answer
308 views

Weight update rule for Rosenblatt's Perceptron

I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron: $\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$ It seems ...
user avatar
1 vote
1 answer
128 views

Use a multilevel logistic regression and cross validation

I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
user avatar
0 votes
1 answer
44 views

Am I fundamentally misunderstanding the net input dot product w*x

Most books have the notation of a weight vector w and input matrix x: $$ w = \begin{bmatrix} w_1\\...\\ w_D \end{bmatrix}, x = \begin{bmatrix} x_{11}&...&x_{1D}\\ ...&...&...\\ x_{N1}&...
user avatar
  • 105
1 vote
1 answer
114 views

Equivalence of Deep Feed-Forward Neural Network to Single-Layer Network

I am new to Deep Learning, and I was going through some lecture notes I found online. It said that a feed-forward neural network with $n$ hidden layers and only linear activation functions is ...
user avatar
  • 21
2 votes
0 answers
20 views

Can we get the input from a multilayer perceptron based on the output of one of its hidden layers?

I was reading a relatively new paper that proposed to split a nerual networks layers into groups and sending each group to different nodes to train them in a distributed manner. In order to not send ...
user avatar
4 votes
2 answers
1k views

Can non-linearly separable data always be made linearly separable?

A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
user avatar
  • 141
1 vote
1 answer
523 views

Kernelized perceptron algorithm weights update

I'm asked to find the maximum margin decision surface separating positive from negative samples by inspection. The positive examples are (1,1) and (-1,-1), the negative ones are (1,-1) and (-1,1). The ...
user avatar
2 votes
3 answers
1k views

Can a Single Layer Perceptron Learn a Nonlinear Function?

A lot of times I've read that with a Single Layer Perceptron (SLP), you can only learn linear functions. But what if we use a generalized linear model? Imagine we only have one input feature $x_1$. ...
user avatar
2 votes
1 answer
346 views

Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
user avatar
1 vote
2 answers
322 views

LabelEncoder with a Multi-Layer Perceptron?

So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline: ...
user avatar
  • 207
1 vote
2 answers
798 views

Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?

I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. ...
user avatar
8 votes
1 answer
494 views

Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min \dfrac{1}{2}||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss ...
user avatar
1 vote
1 answer
32 views

Why do my training losses go up?

I am new to Machine Learning and Tensorflow. For one of my courses, I need to train an MLP for the xor gate. But my losses somehow go up each epoch, which confuses me and I must admit that I ran out ...
user avatar
2 votes
1 answer
552 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
user avatar