Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

0
votes
0answers
19 views

Writing equation of Neural Network model

I am very new to neural networks. My goal is to replace my multivariate regression model with NN and use its output to perform mathematical optimization in order to identify the values of the input ...
-1
votes
0answers
22 views

Some Recent Mathematically Rigorous Deep Learning Papers [on hold]

I am looking for some mathematically rigorous recent deep learning papers. Can someone suggest some?
2
votes
0answers
20 views

Accuracy of Keras Model is Very Low for Identifying Differently Colored Objects

I am using transfer learning approach to train my keras model to identify objects which have same structure but the colors are different i.e objects are to be identified by their respective color. ...
0
votes
0answers
22 views

Are there any optimizers that perform well on small datasets?

With regards to neural networks (or any optimization-based model in general), are there any optimizers that excel in the small dataset regime? Most optimizers I know of require large amounts of data ...
0
votes
1answer
12 views

On evaluating variational autoencoders with prior likelihood and reconstruction error

A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
0
votes
0answers
22 views

why the accuracy of my CNN decresing after some epochs?

at high accuracy, after some epochs the accuracy as well as validation accuracy is decreasing and got stuck after few more epochs. i dont understand why this happened. does more epochs at some point ...
0
votes
1answer
10 views

How is Bidirectional-RNN different from vanilla RNN with adding reverse copy of input?

I have several questions regarding Bi-RNN. The RNN here can be LSTM or GRU as well. (1) What is the input of Bi-RNN when making inference? For RNN, if I want to predict a $\hat{y}(t)$ for the target $...
1
vote
2answers
21 views

What is element-wise max pooling?

I came across this term in the VoxelNet paper in relation to point cloud based object detection using machine learning. It is mentioned in figures 2&3 and in 2.2.1 I am familiar with 2d max ...
0
votes
0answers
23 views

Target for a Poker game Neural Network

I've created a python simulation of a texas holdem poker game with 3 random actions, check/fold, check/call and bet. I have then looped this round several times to produce a training data set for each ...
0
votes
0answers
21 views

Neuronal Network to approximate function from training samples [duplicate]

I'm trying to implement a neuronal network that approximates a certain function, although the term "function" here is mathematically probably imprecise and wrong. Anyway, here's the idea. I have a ...
0
votes
1answer
29 views

How to calculate the lag of a prediction of a time series?

I am trying to learn a time series (Mackey-Glass) using a neural net. In order to see if there has been success in the learning process, I am looking at the correlations between the predicted and real ...
0
votes
1answer
9 views

Evaluation network performance using cross-validation

Suppose I have a data set on which I'm training a neural network. I'm using four-fold validation, meaning that I train four models, one for each fold. Two of the folds are used for training, one for ...
4
votes
1answer
300 views

Is stochastic gradient descent pseudo-stochastic?

I know that stochastic gradient descent randomly chooses 1 sample to update the weights. An epoch is defined as using all $N$ samples. So with SGD, for each epoch, we update the weights $N$ times. ...
0
votes
0answers
7 views

Convolution neural networks vs Capsule networks

My question is more theoretical. Let's say I have 5x11 matrices where each row corresponds to some summary statistic. And each column refers to some subwindow of a large window. The matrix consists ...
0
votes
0answers
16 views

Time series forecasting using Neural Network in matlab [on hold]

Please help me to create neural network to forecast inflation in Matlab for my research.
0
votes
0answers
7 views

How should I understand a Self-Organizing Fuzzy Neural Network?

I'm currently doing research to write a paper for a conference submission (undergraduate-level) and had a question regarding the research I've been conducting. My topic is on using Twitter sentiment ...
0
votes
0answers
18 views

Goodness of fit test for any regression model?

Is there a general goodness-of-fit test for any kind of regression model? My problem is that I have a deep neural network that tries to predict some real value labels using high-dimensional input. The ...
0
votes
1answer
12 views

Do forward activations tell you anything in neural networks?

If you plot the forward activations over time for each layer, are there any patterns that could explain why the network isn't converging, slow learning, etc?
0
votes
1answer
18 views

Theoretical results regarding the size of the training set for neural networks

Have you ever seen anything like a theoretical approach for determining the optimal size (or perhaps some bounds for it) of the training set for a neural network? I know that this is a very broad ...
0
votes
0answers
22 views

Neural Networks cannot learn noisy 1D correlation

I have a noisy 1D problem that the neural network simply cannot discriminate. It is just as good as random chance. My dataset is at http://s000.tinyupload.com/?file_id=75528637079351980231 col0 is ...
0
votes
0answers
12 views

May negative dataset cause CNN model over/under-fitting?

To put you into context, let me explain a bit what I'm trying to achieve. I'm using YOLOv3 (doesn't really matter now) convolutional neural network to detect traffic signs in full images. I'm training ...
0
votes
0answers
5 views

On the practical usage of generalisation error bounds

(This questions is based on a question that I've posted previously here, but I would like it to get more exposure) In many practical scenarios, one would like to answer how much more data is needed ...
0
votes
0answers
18 views

1-D CNN vs DNN performance

I have three datasets of sizes $7065 \times 89$, $14364 \times 89$, and $21432 \times 89$. I have build a neural network that contains one 1-D convolution layer and 1 fully connected layer that goes ...
0
votes
0answers
21 views

Combining a neural network and hidden Markov models

I am reading a paper where authors use neural networks to produce emission and transition probabilities. And I am confused about the way they've described their emission architecture and transition ...
3
votes
1answer
21 views

How to perform cross validation in semi-supervised learning

Suppose in semi-supervised learning, we have labeled set $X_L$ and unlabeled set $X_U$ Is it ok to validate model performance on labeled data only? How to do cross-validation in transductive learning,...
4
votes
1answer
28 views

Is background subtraction common practice for image classification?

I am going to build a mushroom identification application and using neural networks for image classification. Right now I am thinking about different image processing methods to implement before ...
5
votes
2answers
490 views

How well should I expect Adam to work?

I've been coding up a neural network package for my own amusement, and it seems to work. I've been reading about Adam and from what I've seen it's very difficult to beat. Well, when I implement the ...
3
votes
2answers
48 views

What are the benefits of layer-specific learning rates?

I've read about using different learning rates for different layers of neural networks instead of using the same global learning rate for each layer. What's the need for using these different ...
0
votes
0answers
12 views

Model that fits ellipse for pupil detection

I have an eye tracking system which fails to correctly detect the pupil in the camera image when there is a larger luminance gradient across it. This happens if the pupil is very dilated. For me it's ...
0
votes
1answer
32 views

Defining the cases where Neural Networks outperform tree-based methods

It is well-known that neural networks are currently superior to most of the alternatives to do prediction from images (with CNNs) and sequential data (RNN, transformers...). However for other ...
1
vote
1answer
30 views

Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts. Now, I am ...
0
votes
0answers
7 views

How to calculate precision-recall curve for multiple binary classifier for multiple classes

I am using autoencoder based multiple binary classifier for text regeneration each trained on data related to multiple classes. In other words, each classifier is trained to classify only one class. I ...
1
vote
1answer
26 views

Neural Networks - Difference between 1 and 2 layers?

I'm currently working on a regression problem, using neural networks to constrain parameters for a complex physical scenario. I am searching the hyperparameter space for the best model and have thus ...
0
votes
0answers
13 views

How can I fine tune simple RNN or LSTM? [closed]

I'm dealing with RNN and LSTM models by normalized data in range of [-1,+1] and reshaped data for each time sequence from 3 individual matrices A,B,C to long row includes elements of all 3 matrices ...
1
vote
1answer
34 views

How to use classifier a to test whether the boundary between classes (and the classes, themselves) is (are) similar across two datasets

I have two datasets which each contain two classes (four classes total). I suspect that both datasets contain the same kinds of classes (i.e. the boundary that distinguishes classes is similar across ...
0
votes
0answers
10 views

What might explain inferior performance in a LSTM featuring a convolution layer?

This is not a problem:solution scenario insofar that I am not attempting to find a way to improve the model, merely find a reason for its behaviour. Model using LSTM has accuracy of about 84-86% ...
0
votes
1answer
14 views

Sample complexity of deep reinforcement learning agents on smaller state spaces versus zero-padded state spaces

If I train two agents, one on environment A and one on environment A', where A' is just environment A padded with 10 rows of zeros, what can I predict will happen in terms of relative sample ...
0
votes
3answers
47 views

Neural Network for input values optimization

I have electric machine, which parameters I measure by 10 sensors. 8 of them measures "input" values and 2 of them result (output). I've got tons of historical data of all of these sensors. I built a ...
4
votes
2answers
68 views

What are the best books to study Neural Networks from a purely mathematical perspective?

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, ...
1
vote
0answers
13 views

Multiple target regression models statistical testing

I trained two different neural networks with multiple outputs (for solving inverse problem) and it is obvious that one model is better than another but I would like to confirm that there is ...
0
votes
3answers
49 views

Weights not converging while cost function has converged in neural networks [closed]

I'm talking in an ideal scenario where a validation set isn't used. Without validation, as many epochs as possible are calculated. Training stops and finishes only when the loss function is minimized ...
0
votes
0answers
9 views

Why is it most energetically favourable for feature extraction to occur in energy-based models like RBMs?

My current understanding of Restricted Boltzmann Machines (RBMs) is as follows. Please correct me if I'm wrong, as misunderstanding RBMs may be the cause of my question. An RBM is an energy-based ...
1
vote
0answers
18 views

How to plot the gradient descent of a RNN model built using keras? [closed]

I'm exploring how an LSTM solves the problem of vanishing gradients. I have created a simple LSTM model on keras. I know that model.fit() returns a history object that stores model loss and accuracy ...
0
votes
0answers
8 views

Offline signature verification - deep learning

I am looking for good articles to read about offline signature verification using deep learning. maybe even a 'dumbbed down' version for someone who is new to deep learning and convolutional neural ...
0
votes
1answer
14 views

Encoding Layers in the Transformer

In the transformer architecture for NLP, at each layer there are multiple self-attention filters. My question is about the encoded content within these filters. An example can be found here: My ...
3
votes
1answer
34 views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
3
votes
1answer
50 views

what exactly happens during each epoch in neural network training

1.Across different epochs, which of the following is/are updated? initial weights (initial ConvNet filter matrices, initial fully connected weights) hyper parameters: number of ConvNet filters, size ...
0
votes
1answer
38 views

How to detect if model is overfitting?

I know this question is asked billion times, but I could not really find an answer to my situation. So, I want to show all the logs of Keras model learning. The problem is I don't know if my model is ...
0
votes
0answers
13 views

How to return predicted values to original scale after using diff() then modeling using nnetar() functions

I have a series which is a non-stationary series. I analyzed its stationary using Augmented Dickey–Fuller test. I'd like to make a prediction model using nnetar() function in forecast package for my ...