All Questions
Tagged with perceptron neural-networks
94 questions
1
vote
0
answers
22
views
Why is the threshold term incorporated into the weight vector in linear classifiers?
In the context of linear classifiers, such as the perceptron or logistic regression, I understand that the decision boundary is defined by a linear combination of input features and weights, plus a ...
- 111
0
votes
0
answers
18
views
Find weights and bias of Discrete Perceptron
I am studying for an exam and came across this question which I am unable to solve.
How do I find the corresponding weights and biases.
I understand that it is a perceptron so the activation function ...
- 163
0
votes
0
answers
15
views
Trouble understanding why adaline works
Recently came across adaline (an improvement on perceptron) but I am having trouble understanding why adaline works.
Lets take an example of 2D binary classification task. Assume line 'l' is a linear ...
5
votes
2
answers
177
views
Confusion regarding the criteria for defining a ML model as a linear model
I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent :
A model is linear
Output class label is a linear ...
- 53
0
votes
1
answer
58
views
Is it possible to predict non-classifiable label using single layer perceptron and sigmoid function? (without using any perceptron library)
Imagine predicting BMI index like 1,2,3,4,5 and having weight and height as input. I know it can be easily done with other method. Also I have to use sigmoid function and I am really new to this. I ...
1
vote
1
answer
50
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
- 11
2
votes
1
answer
511
views
Is bias nothing but perceptron threshold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 151
3
votes
0
answers
56
views
What's the unified definition of Tikhonov regularization
I met with "Tikhonov regularization" in two textbooks. The first is "Pattern Recognition and Machine Learning" by Christopher M. Bishop. In page 267 of his book, the regularized ...
- 333
1
vote
1
answer
963
views
What are the mathematical differences between the Perceptron and the MP-Neuron?
I try to understand the differences between the MP Neuron and the Perceptron. Is my understanding right that the MP Neuron mathematically only differences in the activation function. I.e. the MP ...
- 139
4
votes
2
answers
180
views
Understanding Weight Updates in a Perceptron
I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j
\end{equation} $
Where:
$\begin{equation}
\...
- 41
1
vote
0
answers
160
views
How can I explain the difference in decision boundary and its reason in relation to the dimension of the hidden layer?
I got these 3 different plots of decision boundaries using 3 different parameters for hidden_layer_sizes of the MLPClassifier from sklearn on XOR gate.
...
- 21
2
votes
1
answer
141
views
Explanation of Equation 5.80 in Pattern Recognition and Machine Learning - Bishop
How the equation 5.80 in _Pattern Recognition and Machine Learning_ by Bishop is derived?
- 93
3
votes
1
answer
285
views
Does the n >> p holds also for minibatches
It's pretty known that when dealing with models (without regularization) the main assumption is $n >> p$ where $p$ is the number of features in the dataset
Let's suppose that we have 1.000.000 ...
- 1,381
4
votes
1
answer
172
views
How do I check if the weights of my perceptron/step activation function are correct
I am new to stack overflow and deep learning so I hope I am doing this the right way. I tried to find the solution myself but it has not been successful so I am seeking some help. This is the ...
- 43
1
vote
0
answers
1k
views
Neural network for imbalanced data
I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
- 45
1
vote
0
answers
76
views
Differentiating output of layer with respect to its input [duplicate]
Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$.
I was ...
- 11
0
votes
0
answers
49
views
Why do the training and validation accuracy drop after tuning?
I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped.
Then someone told me that I should add Dropout layers with 50% dropping.
I did that. ...
- 2,125
2
votes
1
answer
313
views
Single-layer perceptron mathematical formulation
I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then
$$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\...
- 659
0
votes
1
answer
169
views
Did Hinton introduce the concept of distributed representation?
From Goodfellow et al.'s Deep Learning book:
Several key concepts arose during (...) the 1980s that remain central to today’s deep learning. One of these concepts is that of distributed ...
- 169
20
votes
1
answer
6k
views
Why pure exponent is not used as activation function for neural networks?
The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU).
The overall idea of these functions is the same: before ...
- 303
2
votes
1
answer
244
views
How to resolve the perceptron dilemma for binary classification?
I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
- 734
3
votes
1
answer
121
views
Best approach for energy demand forecasting
I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this:
...
- 53
3
votes
1
answer
129
views
About Multi-layer Perceptrons
I've always been a bit confused when it comes to Deep Learning terminology.
Is the definition of the perceptron, whether single layer or multi layer, associated with a specific type of activation ...
- 101
1
vote
0
answers
98
views
Rule of thumb for Data Requirements when Designing a Neural Network for Deep Learning
I'm designing an MLP classifier and I've been noticing that:
Using a very shallow network, or one whose at least one layer has a small number of neurons yields bad performance
Using a deep network ...
- 111
1
vote
0
answers
95
views
Why do we use Matrix in Perceptron instead of Functions?
Matrices are good objects to store connections between dimensions/entities. However, matrix computation is often time consuming and sometimes wasteful if matrix is too sparse. Also thinking about the ...
- 111
2
votes
1
answer
852
views
Whats the different between Logistic regression and perceptron?
In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in
classification results, since they will have the same decision ...
- 21
1
vote
1
answer
307
views
Can we express CNNs in terms of a MLP?
I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
8
votes
1
answer
2k
views
Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?
In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where
MLP neuron outputs $\...
1
vote
1
answer
2k
views
Neural network regression with constraint
I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall ...
- 4,487
2
votes
1
answer
386
views
Can we use perceptron training algorithm to train a single neuron with sigmoid activation?
The perceptron training algorithm is summarized as:
Apply the inputs and calculate the output $ y $
Compare with the desired output yd and calculate error $e = y-y_d$
Update the weights based on the ...
- 119
5
votes
2
answers
2k
views
SciKit Learn: Multilayer perceptron early stopping, restore best weights
In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
- 747
0
votes
1
answer
773
views
Weight update rule for Rosenblatt's Perceptron
I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron:
$\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$
It seems ...
1
vote
1
answer
527
views
Equivalence of Deep Feed-Forward Neural Network to Single-Layer Network
I am new to Deep Learning, and I was going through some lecture notes I found online. It said that a feed-forward neural network with $n$ hidden layers and only linear activation functions is ...
- 23
2
votes
0
answers
82
views
Can we get the input from a multilayer perceptron based on the output of one of its hidden layers?
I was reading a relatively new paper that proposed to split a nerual networks layers into groups and sending each group to different nodes to train them in a distributed manner. In order to not send ...
- 121
2
votes
1
answer
885
views
Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron
I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane.
$w=\sum\limits_{i\in I}^{} y_i x_i$
For reference, I'm studying from the ...
- 45
1
vote
2
answers
857
views
LabelEncoder with a Multi-Layer Perceptron?
So we're working on a machine learning project at work and it's the first time I'm working with an actual team on this. I got pretty good results with a model that uses the following SKLearn pipeline:
...
- 217
1
vote
2
answers
1k
views
Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?
I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. ...
1
vote
1
answer
36
views
Why do my training losses go up?
I am new to Machine Learning and Tensorflow.
For one of my courses, I need to train an MLP for the xor gate.
But my losses somehow go up each epoch, which confuses me and I must admit that I ran out ...
3
votes
1
answer
1k
views
Convergence speed of perceptron algorithm
I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
- 33
1
vote
1
answer
1k
views
Can a unit-step function introduce non-linearity if used in the hidden nodes?
I have a follow up from this question. From the answers in that question, it's obvious that at minimum, a shallow network (only has a single hidden layer), as defined in the "Universal Approximation ...
- 1,932
0
votes
1
answer
227
views
linear perceptron algorithm with 3 training samples
So I am working on a linear perceptron algorithm problem that has 3 training samples. (2D space)
...
2
votes
1
answer
7k
views
Gradient descent with Binary Cross-Entropy for single layer perceptron
I'm implementing a Single Layer Perceptron for binary classification in python. I'm using binary Cross-Entropy loss function and gradient descent.
The gradient descent is not converging, may be I'm ...
- 193
2
votes
1
answer
1k
views
Rule of thumb Overfit in a MLP or is it possible with N = 135
I want to say ahead that I highly appreciate any Literature recommendation (Book, blogg, ...) besides ESL and ISL.
My Question:
Is it possible to train a 3 layer multilayer perceptron (mlp) in a ...
- 425
2
votes
2
answers
18k
views
How are the weights updated in the perceptron learning rule?
I'm considering a perceptron model. I know that when feeding observations from the training dataset to the model, if the model correctly classifies the input, then the weights for this input will not ...
- 1,256
5
votes
2
answers
3k
views
Why perceptron is linear classifier?
It is said that perceptron is linear classifier, but it has a non-linear activation function f = 1 if wx - b >= 0 and f = 0 otherwise
If i will use some non-linear function on linear combination of ...
- 71
1
vote
1
answer
33
views
Training Perceptrons with Backprop
Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule?
is it ...
- 13
0
votes
0
answers
73
views
Choosing perceptron weights to achieve 0% error
I'm really not sure what to do for this question, although I think I would be able to do q3 if I knew how to do q2
user243609
1
vote
1
answer
91
views
Neural Networks Perceptrons and MLPs
While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts.
Now, I am ...
- 69
0
votes
1
answer
8k
views
Neural Network Weight Update
I posted this on Software Engineering and was told that it might be better here and on AI. I'm currently reading Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky (3rd ...
- 3
3
votes
1
answer
800
views
Can this network learn the XOR function?
Let's say I have the following constraints:
The architecture is fixed (see image) (note that there are no biases)
Activation function for the hidden layer is ReLU
There's no activation function for ...
- 39