All Questions
Tagged with perceptron machine-learning
79 questions
1
vote
0
answers
22
views
Why is the threshold term incorporated into the weight vector in linear classifiers?
In the context of linear classifiers, such as the perceptron or logistic regression, I understand that the decision boundary is defined by a linear combination of input features and weights, plus a ...
- 111
1
vote
2
answers
90
views
ADALINE simple implementation with 2 features bug
I am reading Machine Learning with PyTorch and Ski-kit learn book by Sebastian Raschka
While plotting the decision boundary (a line in this case, since the number of features considered = 2) I can't ...
- 21
0
votes
0
answers
18
views
Find weights and bias of Discrete Perceptron
I am studying for an exam and came across this question which I am unable to solve.
How do I find the corresponding weights and biases.
I understand that it is a perceptron so the activation function ...
- 163
0
votes
0
answers
15
views
Trouble understanding why adaline works
Recently came across adaline (an improvement on perceptron) but I am having trouble understanding why adaline works.
Lets take an example of 2D binary classification task. Assume line 'l' is a linear ...
5
votes
2
answers
177
views
Confusion regarding the criteria for defining a ML model as a linear model
I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent :
A model is linear
Output class label is a linear ...
- 53
0
votes
1
answer
58
views
Is it possible to predict non-classifiable label using single layer perceptron and sigmoid function? (without using any perceptron library)
Imagine predicting BMI index like 1,2,3,4,5 and having weight and height as input. I know it can be easily done with other method. Also I have to use sigmoid function and I am really new to this. I ...
4
votes
1
answer
87
views
Why there are two different versions for batch perceptron algorithm?
In the book "Understanding Machine Learning, S. David Ben et al.", the authors describe the Batch Perceptron Algorithm as follows:
However, in the book "Python Machine Learning, ...
- 670
2
votes
0
answers
115
views
Proving Perceptron algorithm mistake bound is tight [closed]
How would I prove the Perception mistake bound is tight. Avrim Blum’s lecture notes claim that the upper bound for mistakes is $\frac{R}{\gamma}^2$, but I don’t understand how to prove this is mistake ...
- 21
1
vote
1
answer
50
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
- 11
2
votes
1
answer
511
views
Is bias nothing but perceptron threshold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 151
4
votes
2
answers
180
views
Understanding Weight Updates in a Perceptron
I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j
\end{equation} $
Where:
$\begin{equation}
\...
- 41
1
vote
0
answers
160
views
How can I explain the difference in decision boundary and its reason in relation to the dimension of the hidden layer?
I got these 3 different plots of decision boundaries using 3 different parameters for hidden_layer_sizes of the MLPClassifier from sklearn on XOR gate.
...
- 21
2
votes
0
answers
63
views
What are the differences between the perceptron and linear regression hypothesis set?
While studying machine learning I have known 2 learning models: linear regression and perceptron.
I know the difference between the Learning algorithm they use, but the hypothesis set look the same to ...
- 81
1
vote
0
answers
76
views
Differentiating output of layer with respect to its input [duplicate]
Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$.
I was ...
- 11
20
votes
1
answer
6k
views
Why pure exponent is not used as activation function for neural networks?
The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU).
The overall idea of these functions is the same: before ...
- 303
2
votes
1
answer
244
views
How to resolve the perceptron dilemma for binary classification?
I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
- 734
0
votes
1
answer
20
views
In AI/ML, using the Perceptron model, would it ever make sense to have both negative weights and data?
I understand the math but I want to make sure I understand the mapping back to real world scenarios. Thinking about it logically, I cannot think of a real world scenario where you would have a ...
- 255
3
votes
1
answer
121
views
Best approach for energy demand forecasting
I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this:
...
- 53
1
vote
1
answer
923
views
How to draw the single perceptron decision boundary when weights and bias are 0?
I've been following an algorithm described on a book called Knowledge Discovery with Support Vector Machines by Lutz H. Hamel. In the book, there is this learning algorithm for a single perceptron ...
- 145
2
votes
1
answer
852
views
Whats the different between Logistic regression and perceptron?
In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in
classification results, since they will have the same decision ...
- 21
1
vote
1
answer
307
views
Can we express CNNs in terms of a MLP?
I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
0
votes
1
answer
773
views
Weight update rule for Rosenblatt's Perceptron
I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron:
$\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$
It seems ...
1
vote
1
answer
217
views
Use a multilevel logistic regression and cross validation
I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
2
votes
0
answers
82
views
Can we get the input from a multilayer perceptron based on the output of one of its hidden layers?
I was reading a relatively new paper that proposed to split a nerual networks layers into groups and sending each group to different nodes to train them in a distributed manner. In order to not send ...
- 121
4
votes
2
answers
3k
views
Can non-linearly separable data always be made linearly separable?
A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
- 141
2
votes
3
answers
3k
views
Can a Single Layer Perceptron Learn a Nonlinear Function?
A lot of times I've read that with a Single Layer Perceptron (SLP), you can only learn linear functions. But what if we use a generalized linear model?
Imagine we only have one input feature $x_1$. ...
- 31
1
vote
2
answers
857
views
LabelEncoder with a Multi-Layer Perceptron?
So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline:
...
- 217
1
vote
2
answers
1k
views
Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?
I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. ...
1
vote
1
answer
36
views
Why do my training losses go up?
I am new to Machine Learning and Tensorflow.
For one of my courses, I need to train an MLP for the xor gate.
But my losses somehow go up each epoch, which confuses me and I must admit that I ran out ...
3
votes
1
answer
1k
views
Convergence speed of perceptron algorithm
I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
- 33
0
votes
1
answer
227
views
linear perceptron algorithm with 3 training samples
So I am working on a linear perceptron algorithm problem that has 3 training samples. (2D space)
...
2
votes
1
answer
1k
views
Why does Perceptron use L1 norm as its error function?
Usually, people use L2 norm as a machine learning error function.
per wiki
the error function for a Perceptron model is
${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|}$
why is that?
- 2,005
2
votes
2
answers
18k
views
How are the weights updated in the perceptron learning rule?
I'm considering a perceptron model. I know that when feeding observations from the training dataset to the model, if the model correctly classifies the input, then the weights for this input will not ...
- 1,256
1
vote
1
answer
33
views
Training Perceptrons with Backprop
Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule?
is it ...
- 13
1
vote
1
answer
276
views
No need for bias term if data is standardised? Linear classification models
For linear classification models, e.g. perceptron, bias term allows to move separating hyperplane away from origin. If data is scattered around the zero does that mean that we don't need bias term?
- 59
0
votes
0
answers
73
views
Choosing perceptron weights to achieve 0% error
I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
user243609
3
votes
1
answer
800
views
Can this network learn the XOR function?
Let's say I have the following constraints:
The architecture is fixed (see image) (note that there are no biases)
Activation function for the hidden layer is ReLU
There's no activation function for ...
- 39
3
votes
1
answer
7k
views
Derivation of Perceptron weight update formula
I've started out studying Machine Learning and am currently reading up about how a single perceptron works. From the wikipedia page, my understanding is as follows: suppose we have an input sample $\...
- 750
0
votes
1
answer
59
views
Are basic multilayer perceptrons well-suited to prediction of non-independent events?
Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality.
Less cryptically put, MLP's are great ...
2
votes
1
answer
917
views
What is the expression for derivative of the signum function one should use in the BP training method
The back propagation learning method requires knowing of derivatives of activation functions. But what expression one should use for signum activation function
$$
\mathrm{Signum}( x ) =
\...
- 183
4
votes
1
answer
4k
views
What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?
As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ?
I ...
- 385
3
votes
1
answer
456
views
MLP - what did author have in mind?
I am currently going through Machine learning and pattern recognition and I have following dilemma:
A algorithm for MLP is presented in that way
and corresponding python code to equation 4.9 (which ...
- 143
2
votes
1
answer
3k
views
How can a perceptron be used for regression?
I know perceptron is a binary classifier which has a 0/1 output. But in one of my exercises for a Neural Network course, there is a question that asks to implement a linear regression with perceptron. ...
- 307
5
votes
0
answers
2k
views
Adding regularization term to perceptron weight update
I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes...
Usually the regularization term is shown in the ...
- 51
0
votes
1
answer
238
views
Clarification on simple perceptron neural network
I understand the title (and the question itself) is a little generic, but I have some questions that I doubt I can find from google search or studying the topic (yes, I've tried for a while).
...
- 103
1
vote
1
answer
216
views
Intuition behind neural networks
I'm really interested in understanding the intuition behind multilayer perceptrons and neural networks.
I'm following the Caltech video which is excellent
https://www.youtube.com/watch?v=Ih5Mr93E-...
- 571
1
vote
2
answers
9k
views
What is weights in perceptron
I am just diving into machine learning and started with learning artificial neural networks. So on learning about perceptron I stucked on wording "weights".
Is it rate of how much input item matched?
...
- 111
1
vote
1
answer
671
views
Why does the weight vector in a perceptron monotonically tend to the generously feasible region
In a course on Machine Learning, in the chapter about a Perceptron, there is this statement:
If a generously feasible region exists, then the distance between the current weight vector and a weight ...
- 1,199
1
vote
1
answer
343
views
Approximating SVM using Perceptron
Suppose that we have a set of linearly separable data and this pseudocode of Perceptron:
...
- 11
0
votes
3
answers
1k
views
Does the adjustment / learning of the weights in Perceptron algorithm depend on the learning rate?
For perceptron algorithm, the output and target values are either $0$ or $1$.
Assume output is $y$ and target is $d$.
From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can ...
- 175