Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

Total Loss Going Up after First Checkpoint with LayoutLM

I'm trying to finetune LayoutLM V3 Base model using the provided dit/train_net.py script on my own custom dataset that is similar to PubLayNet. The learning starts ...
user avatar
0 votes
0 answers
6 views

How to deal with unknown classes with a convolution neural network classifier?

I'm quite new into the DL and ML field. I'm training a CNN able to classify 3 different classes, however I would like in the testing phase to make the CNN able to not misclassify images that do not ...
user avatar
  • 1
0 votes
0 answers
8 views

Improve the perfomance of the deep learning model based on the train and validate loss curve [duplicate]

I have a deep learning model and the following is the loss on the train and validate data. The prediction for my model is not good. Do you know what I should do for my model to have a better results? ...
user avatar
  • 13
1 vote
1 answer
13 views

Do I need to normalize data before applying L1, L2 norm in ANN

I wish to train the ANN and use regularizers to avoid overfitting. I need some suggestions, is it mandatory to normalize the data before using L1, L2 regularizers. I would highly appreciate if you can ...
user avatar
  • 141
0 votes
0 answers
7 views

Concatenation or separate channels for a CNN

let's say I am classifying time series data from multiple channels in a biomedical setup (e.g. 12 lead ECG). I have been reading this paper on a CNN-based (ResNet) architecture for assesing the ...
user avatar
  • 101
0 votes
0 answers
2 views

Does the attention mechanism (in CNNs) bring additional parameters/weights to learn to the network?

The idea of the attention mechanism is based on using some weighted sum of the output of some layers in deep networks. I see the process in forward propagation, and it seems that the attention ...
user avatar
  • 189
0 votes
0 answers
14 views

Dependent variable standardization in neural networks

I am using a multilayer perceptron model to predict urban temperatures. I have standardized the independent variables before training the model. However, I have not standardized the dependent variable....
user avatar
0 votes
0 answers
12 views

How can I get the Binary Cross Entropy from the Cross Entropy function for GANs

I got the definition of log-likelihood by Goodfellow's Deep Learning book: \begin{equation} \label{eq:loglikelihood} \theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta). \end{...
user avatar
0 votes
0 answers
17 views

Statistical test when comparing oversampling to no oversampling on ANN

I use 70% of the dataset for training and 30% for testing. I use oversampling on the training dataset with an ANN. I use the test dataset on my ANN and look at the performance of oversampling against ...
user avatar
0 votes
0 answers
24 views

Can somebody help me understand the sentences in more readable expresions?

I was reading a paper about "bayesSimIG" and I have problem in understand the following paragraph.I have read it many times and did a lot of research for it and have understood what each ...
user avatar
  • 2,585
0 votes
0 answers
19 views

Neural Network $\delta^L$ is very often zero [closed]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it, I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
15 views

Neural Network not learning [duplicate]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
8 views

How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]

I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
user avatar
0 votes
0 answers
9 views

Using Inception and FID scores in training?

Is it possible to use the Inception and FID scores in the training of a deep image generation model, i.e. to maximize the scores in a loss function, albeit this is "cheating"? If so, has ...
user avatar
  • 1
0 votes
1 answer
19 views

Finding the position of the global optimum with Pytorch [closed]

I have a dataset with 22 parameters and I did a PolynomialFeatures = 2 to find the influence of interaction. This was then fitted to an Artificial Neural Network with the lowest loss being around 0.02....
user avatar
  • 21
0 votes
0 answers
11 views

Shrinkage / L1 regularization as a loss term versus a constraint (post-process step) with momentum optimizers

I have a complex model with very non-linear operations (divisions, exponentials, matrix inversions, square roots, Cholesky decompositions, etc...) for which I want to optimize the parameters. However, ...
user avatar
2 votes
1 answer
36 views

Should I join train and validation sets for final NN model training? If yes, when to stop training the final model?

Normally we divide our dataset into 3 sets: train set, validation set, test set. We use train set to find optimal parameters (weights and biases of NN) and validation set to find optimal NN ...
user avatar
1 vote
0 answers
10 views

is nrmse scale-dependent?

Im trying to evaluate my regression models using a normalised version of the RMSE, nrmse = rmse(y, y_pred)/rmse(y, y_mean) where y_mean is the array of the same len as y filled with the mean value of ...
user avatar
  • 11
1 vote
0 answers
10 views

Encoding Geolocation data

I am working on routing bus from one stop to another, for which Geolocation data inform of latitude and longitude is required. In addition to xy coordinates, distance matrix of locations is also ...
user avatar
0 votes
0 answers
16 views

Best way to approximate head point having only face keypoints

I'm using the BlazeFace model from TensorFlow which only has this few keypoints: I need those keypoints plus a head keypoint, like this one: My question is, which would be the best way to ...
user avatar
0 votes
0 answers
7 views

Validation Loss for my binary image classifier model is increasing. how to bring it down? [duplicate]

I am new to the domain of Deep learning and I have been trying to create a binary image classifier using a dataset which I created by myself. I am building the model from scratch. It is CNN model. ...
user avatar
  • 1
0 votes
0 answers
20 views

How is the extrapolation variant of the parity problem defined?

In the paper PonderNet: Learning to Ponder (Banino et al. 2021), the authors define the following "Parity" task: input vectors had 64 elements, of which a random number from 1 to 64 were ...
user avatar
1 vote
1 answer
12 views

In a tranformer, the same word can have different attention weights in different sentences?

I'm trying to understand the transformer architecture for NLP. The main issue is regarding the attention weights. The same word can have different attention weights in different sentences, right?
user avatar
  • 378
1 vote
0 answers
11 views

How to tell a conditional generative model is overfitting or not? And how to split training and validation set?

I'm now working on conditional generative models, but it confused me in the training process. My process: split training and validation set, and a held-out test set train the model and evaluate the ...
user avatar
0 votes
0 answers
12 views

How to properly mask MultiHeadAttention for sliding window time series data

I have data in the shape (batch, seq_len, features) that is a time series sliding window. In essence, I'm using the most recent ...
user avatar
0 votes
0 answers
19 views

and what if non-linear activation functions give better results than the linear ones?

I had a regression problem with small data set, I solved it with neural networks (MLP, ELM,..) As convention, I used a linear function for output layer, the results were not so good. I tried to change ...
user avatar
  • 153
2 votes
1 answer
31 views

Restrict output range of a neuron based on output of other neurons

I have a neural network with three output neurons $X_1$, $X_2$ and $X_3$ with output range in [-1, 1]. I have many training data split in 80:20 ratio between training & testing sets. While ...
user avatar
  • 363
1 vote
0 answers
25 views

Variance of the predicted temperature

I am predicting the electrical load and I also use the predicted temperature as one of the input feature. For example, I want to predict the electrical for tomorrow. I use the predicted temperature ...
user avatar
  • 13
0 votes
0 answers
17 views

Similar results between feedforward neural networks, recurrent neural network and LSTM for time series data - Is this standard?

Tl;dr: I have trained feedforward neural networks, recurrent neural networks and LSTM networks to predict behaviour of weather temperature. The results are almost all the same (see below). Is this ...
user avatar
  • 111
0 votes
0 answers
12 views

Can we compare rigorously the computing time to evaluate ReLU or other nonlinear smooth activations?

Can we say that, independently of the computer, computing relu and relu' is cheaper than computing f and f' for some other smooth non-linear activation (e.g. logistic, tanh)? If not, what are the ...
user avatar
0 votes
0 answers
27 views

How to calculate Cosine Similarity from Keras model?

I'm trying to make hybrid recommender system that recommends movies to users from Movielens dataset. Its Content part is based on Doc2Vec model from gensim library and its Collaborative Filtering part ...
user avatar
0 votes
0 answers
22 views

How to choose the best recommender system? What evaluation metrics to use?

I want to build a recommender system to suggest similar songs to continue a playlist (similar to what Spotify does by recommending similar songs at the end of a playlist). I want to build two models: ...
user avatar
  • 1
1 vote
0 answers
6 views

Backprop in Residual neural network?

I'm trying to build a Residual neural network with 2 layers , and I'm having difficultiy understanding what are the equations for the backprop for the following : $$ W_{2}x+tanh(W_{1}x+b_1)+b_2 $$ ...
user avatar
0 votes
0 answers
29 views

Why does Adam optimizer with gradient clipping perform better than simple Adam optimizer?

Since Adam optimizer uses the first and second moments of gradients to adapt the learning rate, what purpose does the gradient clipping serve when used with Adam optimizer or any adaptive learning ...
user avatar
  • 151
0 votes
0 answers
7 views

Customizing anomalies for different customers

I have built an LSTM autoencoder model to identify anomalies in time series wifi throughput data for over 100 customers. However, the definition of anomalies is very subjective. E.g. Customer A thinks ...
user avatar
0 votes
0 answers
14 views

How to report neural network results on test data set

I am working on a manuscript for which I have results from a simple neural network that I would like to report. I have both a training dataset and a test dataset. My metrics of interest include root ...
user avatar
  • 185
0 votes
0 answers
7 views

Image data privacy/encryption technique [closed]

I am working on a human identification task where I have images of people (with or without face). One of our concern is data privacy. Data encryption seems quite straight forward when it comes to ...
user avatar
  • 101
0 votes
0 answers
27 views

Comparing current and new version of Autoencoder

I am building an LSTM Autoencoder (unsupervised model) to detect anomalies in a time series dataset. The input is telemetry data from routers and I want to detect anomalies in the throughout of ...
user avatar
0 votes
0 answers
6 views

How to calculate number of learnable parameters in CNN? [duplicate]

How to calculate number of learnable parameters in CNN when only kernel size and number of filters are given? Lets say the kernel size is x and number of filters is y. In that case in which way I ...
user avatar
0 votes
0 answers
21 views

Mini batches and loss in recurrent neural networks (RNNs)

Suppose that we have a sequence $\left\{x^{(k)}\right\}_{k = 1}^{N}$ and that we wish to use a RNN to predict the next element of the sequence given the previous elements of the sequence (e.g., a ...
user avatar
  • 101
0 votes
0 answers
11 views

Model predicts same number for any input on initialization of random weights

My PNAConv (pytorch) network has been having issues with predicting the same exact value for all inputs. Without getting into too many details, I have a broader neural netowrk question. When I ask my ...
user avatar
0 votes
0 answers
13 views

How to predict a mathematical progression with keras

I try the following model for a many-to-many recurrent network: ...
user avatar
0 votes
0 answers
3 views

A question on the projection step in Generic Adaptive Method Setup: $x_{t+1} = \Pi_{\mathcal{F},\sqrt{V_t}} (\hat{x}_{t+1}).$

I am reading the paper "ON THE CONVERGENCE OF ADAM AND BEYOND". In this paper, they proposed the following framework of adaptive methods. I was confused on the last step: $x_{t+1} = \Pi_{\...
user avatar
  • 101
0 votes
0 answers
21 views

Applying time series forecasting model in categorised data

My dataset looks like this ...
user avatar
  • 101
0 votes
0 answers
16 views

Time Series forecasting with multiple non-parallel input

I have a private dataset describing more or less 200 different crowdfunding projects. In such data, I have the trend of donations (so are uni variate) over time for each project. The problem is that ...
user avatar
0 votes
0 answers
6 views

Which network to segment a rectangle on the ceiling of a room (enclosed by joists), and taking advantage of prior knowledge

I am wondering how you guys would approach this problem. Given an image from a camera pointing towards the ceiling of a room (some joists are present), I want to segment the biggest rectangular area ...
user avatar
2 votes
1 answer
25 views

Why is cross entropy loss better than MSE for multi-class classification? [duplicate]

I know there's a lot of material on this, but I'm still struggling to find a scenario where cross-entropy loss is better than MSE loss for a multi-class classification problem. For example, if we have ...
user avatar
  • 41
1 vote
0 answers
13 views

Why do neural networks have layers instead of being a sea of freely connected neurons? [duplicate]

What I mean is, neural networks are neatly sectioned off into layers. But why? Why not just make a sea of neurons that all connect to each other, then let them sort out their weights and connections ...
user avatar
1 vote
0 answers
15 views

Is Gradient Accumulation equivalent to using larger batch sizes?

Gradient accumulation is used to deal with memory limitation by partitioning a large batch size into small chunks. For example, instead of using a batch size of 1024 samples per batch you could use ...
user avatar
0 votes
0 answers
19 views

What does Feed Forward Policy in terms of Reinforcement Learning?

While reading Reinforcement Learning, I saw the term, Feed Forward Policy and the article also says that it does not have memory, what does "it does not have memory" also mean? If possible, ...
user avatar
  • 2,585

1
2 3 4 5
186