Questions tagged [perceptron]
An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.
167 questions
3
votes
2
answers
825
views
How to find multi-layer perceptron weights?
I want to use a multi-layer perceptron to design the following function :
The architecture I want to use is the following one :
What would be $w_i$ weights ? Is there any guide to find them ?
I ...
CommunityBot
- 1
2
votes
1
answer
2k
views
Deriving step size/learning rate in the hinge loss passive-aggressive/perceptron algorithm
Recall the perceptron algorithm:
cycle through all points until convergence
$\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad
> \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ ...
CommunityBot
- 1
5
votes
2
answers
177
views
Confusion regarding the criteria for defining a ML model as a linear model
I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent :
A model is linear
Output class label is a linear ...
- 53
0
votes
0
answers
14
views
Scaling for binary data features
My goal is a binary classification of a sports match between two players, particularly I am concerned with the probabilities of each player winning.
My current dataframe has feature values of [...
- 47
0
votes
1
answer
227
views
linear perceptron algorithm with 3 training samples
So I am working on a linear perceptron algorithm problem that has 3 training samples. (2D space)
...
CommunityBot
- 1
1
vote
2
answers
90
views
ADALINE simple implementation with 2 features bug
I am reading Machine Learning with PyTorch and Ski-kit learn book by Sebastian Raschka
While plotting the decision boundary (a line in this case, since the number of features considered = 2) I can't ...
- 21
1
vote
0
answers
22
views
Why is the threshold term incorporated into the weight vector in linear classifiers?
In the context of linear classifiers, such as the perceptron or logistic regression, I understand that the decision boundary is defined by a linear combination of input features and weights, plus a ...
- 111
1
vote
1
answer
307
views
Can we express CNNs in terms of a MLP?
I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
CommunityBot
- 1
0
votes
0
answers
18
views
Find weights and bias of Discrete Perceptron
I am studying for an exam and came across this question which I am unable to solve.
How do I find the corresponding weights and biases.
I understand that it is a perceptron so the activation function ...
- 163
0
votes
0
answers
15
views
Trouble understanding why adaline works
Recently came across adaline (an improvement on perceptron) but I am having trouble understanding why adaline works.
Lets take an example of 2D binary classification task. Assume line 'l' is a linear ...
0
votes
1
answer
58
views
Is it possible to predict non-classifiable label using single layer perceptron and sigmoid function? (without using any perceptron library)
Imagine predicting BMI index like 1,2,3,4,5 and having weight and height as input. I know it can be easily done with other method. Also I have to use sigmoid function and I am really new to this. I ...
3
votes
1
answer
7k
views
Derivation of Perceptron weight update formula
I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
- 103
1
vote
1
answer
963
views
What are the mathematical differences between the Perceptron and the MP-Neuron?
I try to understand the differences between the MP Neuron and the Perceptron. Is my understanding right that the MP Neuron mathematically only differences in the activation function. I.e. the MP ...
CommunityBot
- 1
4
votes
1
answer
87
views
Why there are two different versions for batch perceptron algorithm?
In the book "Understanding Machine Learning, S. David Ben et al.", the authors describe the Batch Perceptron Algorithm as follows:
However, in the book "Python Machine Learning, ...
- 670
2
votes
0
answers
115
views
Proving Perceptron algorithm mistake bound is tight [closed]
How would I prove the Perception mistake bound is tight. Avrim Blum’s lecture notes claim that the upper bound for mistakes is $\frac{R}{\gamma}^2$, but I don’t understand how to prove this is mistake ...
- 21
1
vote
1
answer
50
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
- 67.1k
15
votes
3
answers
51k
views
Decision boundary plot for a perceptron
I am trying to plot the decision boundary of a perceptron algorithm and I am really confused about a few things. My input instances are in the form $[(x_{1},x_{2}), y]$, basically a 2D input instance (...
- 113
1
vote
1
answer
1k
views
In what way are SGD and the Perceptron learning algorithm very similar?
I'm reading Hands-On Machine Learning and the author states that:
You may have noticed the fact that the Perceptron learning algorithm strongly resembles Stochastic Gradient Descent. In fact, Scikit-...
CommunityBot
- 1
1
vote
0
answers
16
views
Can Perceptron and Naive Bayes classifier create a vertical decision boundary in a two-dimensional graph?
A decision boundary like in the picture.
2
votes
1
answer
511
views
Is bias nothing but perceptron threshold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 4,487
1
vote
1
answer
343
views
Approximating SVM using Perceptron
Suppose that we have a set of linearly separable data and this pseudocode of Perceptron:
...
CommunityBot
- 1
4
votes
2
answers
180
views
Understanding Weight Updates in a Perceptron
I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j
\end{equation} $
Where:
$\begin{equation}
\...
- 1,245
3
votes
0
answers
56
views
What's the unified definition of Tikhonov regularization
I met with "Tikhonov regularization" in two textbooks. The first is "Pattern Recognition and Machine Learning" by Christopher M. Bishop. In page 267 of his book, the regularized ...
- 333
0
votes
2
answers
2k
views
Update weight vector regardless of correctness for perceptron algorithm
For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong ...
CommunityBot
- 1
1
vote
0
answers
160
views
How can I explain the difference in decision boundary and its reason in relation to the dimension of the hidden layer?
I got these 3 different plots of decision boundaries using 3 different parameters for hidden_layer_sizes of the MLPClassifier from sklearn on XOR gate.
...
- 21
2
votes
1
answer
141
views
Explanation of Equation 5.80 in Pattern Recognition and Machine Learning - Bishop
How the equation 5.80 in _Pattern Recognition and Machine Learning_ by Bishop is derived?
- 93
3
votes
1
answer
285
views
Does the n >> p holds also for minibatches
It's pretty known that when dealing with models (without regularization) the main assumption is $n >> p$ where $p$ is the number of features in the dataset
Let's suppose that we have 1.000.000 ...
- 94k
4
votes
1
answer
172
views
How do I check if the weights of my perceptron/step activation function are correct
I am new to stack overflow and deep learning so I hope I am doing this the right way. I tried to find the solution myself but it has not been successful so I am seeking some help. This is the ...
- 58.2k
1
vote
0
answers
1k
views
Neural network for imbalanced data
I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
- 45
2
votes
0
answers
63
views
What are the differences between the perceptron and linear regression hypothesis set?
While studying machine learning I have known 2 learning models: linear regression and perceptron.
I know the difference between the Learning algorithm they use, but the hypothesis set look the same to ...
- 81
1
vote
0
answers
76
views
Differentiating output of layer with respect to its input [duplicate]
Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$.
I was ...
- 334k
2
votes
3
answers
3k
views
Can a Single Layer Perceptron Learn a Nonlinear Function?
A lot of times I've read that with a Single Layer Perceptron (SLP), you can only learn linear functions. But what if we use a generalized linear model?
Imagine we only have one input feature $x_1$. ...
- 1
1
vote
1
answer
708
views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
- 146
11
votes
4
answers
21k
views
Difference between MLP(Multi-layer Perceptron) and Neural Networks?
I am wondering about the differences. Based on my understanding, MLP is one kind of neural networks, where the activation function is sigmoid, and error term is cross-entropy(logistics) error. Looking ...
- 11
1
vote
1
answer
618
views
What is the interpretation of sklearn's linear perceptron coefficients?
I'm stumped as to why this example doesn't do a better job fitting the data, I suspect it has to do with my interpretation of the perceptron object's coefficients. Note that I'm interested in the <...
- 1,479
0
votes
1
answer
773
views
Weight update rule for Rosenblatt's Perceptron
I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron:
$\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$
It seems ...
CommunityBot
- 1
4
votes
2
answers
3k
views
Can non-linearly separable data always be made linearly separable?
A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
- 163
1
vote
1
answer
260
views
Different results for Logistic Regression (wrong) and Perceptron (correct)
To help me with some understanding, I'm trying to learn the Logical AND and Logical OR using Linear Regression trained over the following data:
...
- 7,262
0
votes
0
answers
49
views
Why do the training and validation accuracy drop after tuning?
I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped.
Then someone told me that I should add Dropout layers with 50% dropping.
I did that. ...
- 2,125
45
votes
6
answers
58k
views
What's the difference between logistic regression and perceptron?
I'm going through Andrew Ng's lecture notes on Machine Learning.
The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
- 26.9k
2
votes
1
answer
313
views
Single-layer perceptron mathematical formulation
I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then
$$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\...
- 58.2k
0
votes
1
answer
204
views
How to show that the gradient of the smoothed surrogate loss function leads to perceptron update?
This is about the contents of section 1.2.1 and 1.2.1.1 of the book "Neural Networks and Deep Learning: A Textbook". The link to the sections is here. The question arises from the following ...
- 333
0
votes
1
answer
169
views
Did Hinton introduce the concept of distributed representation?
From Goodfellow et al.'s Deep Learning book:
Several key concepts arose during (...) the 1980s that remain central to today’s deep learning. One of these concepts is that of distributed ...
- 56.2k
3
votes
1
answer
800
views
Can this network learn the XOR function?
Let's say I have the following constraints:
The architecture is fixed (see image) (note that there are no biases)
Activation function for the hidden layer is ReLU
There's no activation function for ...
CommunityBot
- 1
9
votes
1
answer
704
views
Support Vector Machine with Perceptron Loss
Typical support vector classifier uses the following optimization procedure:
$$\min \dfrac{1}{2}||w||^2 + C\sum_{i=1}^N \zeta_i$$
$$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$
$$\zeta_i \geq 0$$
This hinge loss ...
- 9,678
1
vote
2
answers
857
views
LabelEncoder with a Multi-Layer Perceptron?
So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline:
...
CommunityBot
- 1
1
vote
1
answer
260
views
Stop criterion is Infinitive in Perceptron in Sklearn
I read code in book "Hand-on Machine Learning in Sklearn and TensorFlow" by Aurelien Geron
...
- 6,912
0
votes
2
answers
232
views
Why was it ever thought multi layer perceptrons couldn't implement XOR functions?
As I understand it the Perceptrons books helped start the "AI Winter" because the authors claimed that multilayer perceptrons couldn't implement non linearly separable functions like XOR.
However it ...
- 6,912
20
votes
1
answer
6k
views
Why pure exponent is not used as activation function for neural networks?
The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU).
The overall idea of these functions is the same: before ...
- 14.1k
2
votes
1
answer
244
views
How to resolve the perceptron dilemma for binary classification?
I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
- 1,233