Questions tagged [perceptron]
An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.
163
questions
0
votes
0
answers
27
views
Does linear separability with gamma margin guarantee convergence of perceptron algorithm?
I am studying perceptron for the first time. I came across the assumption from online resources that if the data is linearly separable with gamma margin then the perceptron algorithm will converge. Is ...
- 1
2
votes
0
answers
77
views
Proving Perceptron algorithm mistake bound is tight [closed]
How would I prove the Perception mistake bound is tight. Avrim Blum’s lecture notes claim that the upper bound for mistakes is $\frac{R}{\gamma}^2$, but I don’t understand how to prove this is mistake ...
- 21
1
vote
1
answer
28
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
- 11
1
vote
0
answers
15
views
Can Perceptron and Naive Bayes classifier create a vertical decision boundary in a two-dimensional graph?
A decision boundary like in the picture.
2
votes
1
answer
207
views
Is bias nothing but perceptron threshold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 151
0
votes
0
answers
35
views
Non-optimal separating hyperplane?
This is exercise 4.7 from Elements of Statistical Learning by Hastie, Tibshirani and Friedman.
Consider the generalisation of the perceptron learning algorithm with the goal of minimising $D^*(\beta, \...
- 175
2
votes
0
answers
45
views
What's the unified definition of Tikhonov regularization
I met with "Tikhonov regularization" in two textbooks. The first is "Pattern Recognition and Machine Learning" by Christopher M. Bishop. In page 267 of his book, the regularized ...
- 303
1
vote
1
answer
375
views
What are the mathematical differences between the Perceptron and the MP-Neuron?
I try to understand the differences between the MP Neuron and the Perceptron. Is my understanding right that the MP Neuron mathematically only differences in the activation function. I.e. the MP ...
- 129
4
votes
2
answers
80
views
Understanding Weight Updates in a Perceptron
I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j
\end{equation} $
Where:
$\begin{equation}
\...
- 41
0
votes
0
answers
17
views
Questions about update and convergence in Rosenblatt's Perceptron [duplicate]
While studying Rosenblatt's Perceptron algorithm, I have some interpretation problem with the update rule:
𝐰𝑡+1 = 𝐰𝑡+𝜂(𝑦𝑗−𝑦̂ 𝑗)𝐱𝑗
more exactly: how does xj comes into the picture exactly? ...
- 725
0
votes
0
answers
17
views
Could Rosenblatt's Perceptron's Performance Decrease over Iterations?
Recently I studied Frank Rosenblatt's Perceptron algorithm for classification. While I can recall cases from my experience where multilayer perceptrons' performance could incrementally decrease after ...
- 725
1
vote
0
answers
121
views
How can I explain the difference in decision boundary and its reason in relation to the dimension of the hidden layer?
I got these 3 different plots of decision boundaries using 3 different parameters for hidden_layer_sizes of the MLPClassifier from sklearn on XOR gate.
...
- 21
2
votes
1
answer
103
views
Explanation of Equation 5.80 in Pattern Recognition and Machine Learning - Bishop
How the equation 5.80 in _Pattern Recognition and Machine Learning_ by Bishop is derived?
- 93
3
votes
1
answer
283
views
Does the n >> p holds also for minibatches
It's pretty known that when dealing with models (without regularization) the main assumption is $n >> p$ where $p$ is the number of features in the dataset
Let's suppose that we have 1.000.000 ...
3
votes
1
answer
122
views
How do I check if the weights of my perceptron/step activation function are correct
I am new to stack overflow and deep learning so I hope I am doing this the right way. I tried to find the solution myself but it has not been successful so I am seeking some help. This is the ...
- 33
1
vote
0
answers
852
views
Neural network for imbalanced data
I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
- 45
1
vote
0
answers
44
views
What are the differences between the perceptron and linear regression hypothesis set?
While studying machine learning I have known 2 learning models: linear regression and perceptron.
I know the difference between the Learning algorithm they use, but the hypothesis set look the same to ...
- 61
1
vote
0
answers
44
views
Differentiating output of layer with respect to its input [duplicate]
Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$.
I was ...
- 11
1
vote
1
answer
442
views
What is the interpretation of sklearn's linear perceptron coefficients?
I'm stumped as to why this example doesn't do a better job fitting the data, I suspect it has to do with my interpretation of the perceptron object's coefficients. Note that I'm interested in the <...
- 191
0
votes
1
answer
166
views
Different results for Logistic Regression (wrong) and Perceptron (correct)
To help me with some understanding, I'm trying to learn the Logical AND and Logical OR using Linear Regression trained over the following data:
...
- 157
0
votes
0
answers
35
views
Why do the training and validation accuracy drop after tuning?
I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped.
Then someone told me that I should add Dropout layers with 50% dropping.
I did that. ...
- 2,144
1
vote
1
answer
351
views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
- 13
2
votes
1
answer
206
views
Single-layer perceptron mathematical formulation
I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then
$$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\...
- 557
0
votes
1
answer
162
views
How to show that the gradient of the smoothed surrogate loss function leads to perceptron update?
This is about the contents of section 1.2.1 and 1.2.1.1 of the book "Neural Networks and Deep Learning: A Textbook". The link to the sections is here. The question arises from the following ...
- 303
0
votes
1
answer
107
views
Did Hinton introduce the concept of distributed representation?
From Goodfellow et al.'s Deep Learning book:
Several key concepts arose during (...) the 1980s that remain central to today’s deep learning. One of these concepts is that of distributed ...
- 169
1
vote
1
answer
184
views
Stop criterion is Infinitive in Perceptron in Sklearn
I read code in book "Hand-on Machine Learning in Sklearn and TensorFlow" by Aurelien Geron
...
- 113
18
votes
1
answer
5k
views
Why pure exponent is not used as activation function for neural networks?
The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU).
The overall idea of these functions is the same: before ...
- 283
2
votes
1
answer
221
views
How to resolve the perceptron dilemma for binary classification?
I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
- 714
0
votes
1
answer
20
views
In AI/ML, using the Perceptron model, would it ever make sense to have both negative weights and data?
I understand the math but I want to make sure I understand the mapping back to real world scenarios. Thinking about it logically, I cannot think of a real world scenario where you would have a ...
- 235
1
vote
1
answer
814
views
In what way are SGD and the Perceptron learning algorithm very similar?
I'm reading Hands-On Machine Learning and the author states that:
You may have noticed the fact that the Perceptron learning algorithm strongly resembles Stochastic Gradient Descent. In fact, Scikit-...
- 51
1
vote
0
answers
114
views
One-hot encoding in Keras for R [closed]
I am trying to build a binary classifier using a MLP with the Keras package in R.
My question is, why does the package require the labels to be a one-hot vector? For example, the value 1 will be the ...
- 111
3
votes
1
answer
96
views
Best approach for energy demand forecasting
I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this:
...
- 53
1
vote
1
answer
807
views
How to draw the single perceptron decision boundary when weights and bias are 0?
I've been following an algorithm described on a book called Knowledge Discovery with Support Vector Machines by Lutz H. Hamel. In the book, there is this learning algorithm for a single perceptron ...
- 145
3
votes
1
answer
114
views
About Multi-layer Perceptrons
I've always been a bit confused when it comes to Deep Learning terminology.
Is the definition of the perceptron, whether single layer or multi layer, associated with a specific type of activation ...
- 101
1
vote
0
answers
88
views
Rule of thumb for Data Requirements when Designing a Neural Network for Deep Learning
I'm designing an MLP classifier and I've been noticing that:
Using a very shallow network, or one whose at least one layer has a small number of neurons yields bad performance
Using a deep network ...
- 111
1
vote
1
answer
415
views
Perceptron as a Logistic Regression
If by following way single perceptron is made to work like Logistic Regression.
How much correct is it to say that I made perceptron to work as Logistic Regression.
Question came to mind as ...
- 175
1
vote
0
answers
79
views
Why do we use Matrix in Perceptron instead of Functions?
Matrices are good objects to store connections between dimensions/entities. However, matrix computation is often time consuming and sometimes wasteful if matrix is too sparse. Also thinking about the ...
- 111
2
votes
0
answers
156
views
0
votes
0
answers
49
views
Why in gated recurrent unit gates are controlled by only one layer perceptrons?
Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
- 1
2
votes
1
answer
764
views
Whats the different between Logistic regression and perceptron?
In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in
classification results, since they will have the same decision ...
- 21
1
vote
1
answer
178
views
Can we express CNNs in terms of a MLP?
I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
7
votes
1
answer
2k
views
Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?
In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where
MLP neuron outputs $\...
1
vote
1
answer
1k
views
Neural network regression with constraint
I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall ...
- 3,522
2
votes
1
answer
712
views
Can a 4D perceptron be plotted in 2 dimensions?
I am wondering if it is at all possible to plot a 4D perceptron line in 2D.
Obviously, it would be impossible to observe it with all of its original information, but is there a way for me to observe ...
- 23
2
votes
1
answer
336
views
Can we use perceptron training algorithm to train a single neuron with sigmoid activation?
The perceptron training algorithm is summarized as:
Apply the inputs and calculate the output $ y $
Compare with the desired output yd and calculate error $e = y-y_d$
Update the weights based on the ...
- 119
5
votes
2
answers
1k
views
SciKit Learn: Multilayer perceptron early stopping, restore best weights
In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
- 747
0
votes
1
answer
581
views
Weight update rule for Rosenblatt's Perceptron
I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron:
$\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$
It seems ...
1
vote
1
answer
192
views
Use a multilevel logistic regression and cross validation
I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
0
votes
1
answer
127
views
Am I fundamentally misunderstanding the net input dot product w*x
Most books have the notation of a weight vector w and input matrix x:
$$
w = \begin{bmatrix}
w_1\\...\\ w_D
\end{bmatrix},
x = \begin{bmatrix}
x_{11}&...&x_{1D}\\
...&...&...\\
x_{N1}&...
- 105
1
vote
1
answer
307
views
Equivalence of Deep Feed-Forward Neural Network to Single-Layer Network
I am new to Deep Learning, and I was going through some lecture notes I found online. It said that a feed-forward neural network with $n$ hidden layers and only linear activation functions is ...
- 21