Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
1 vote
0 answers
9 views

Using Variance of time series as input feature for time series clustering

I have a time series dataset, it is a data frame with 2000 rows and 1000 columns. Each rows is for one specific id and has a unique pattern. I want to clustering this data into multiple classes. Let ...
user avatar
  • 23
0 votes
0 answers
6 views

Hyper parameters to tune in Bayesian network?

In tree based models and neural networks, we can optimised the models by tuning the hyper parameters(such as: learning rate, number of neutrons.. etc). Is there a hyper parameters to tune in Bayesian ...
user avatar
  • 1
0 votes
0 answers
19 views

How does attention add expressive power to encoder-decoders?

I am learning about the attention mechanism for the first time, and the context in which it has been introduced (watching Lecture 8 of Stanford's CS224N) is that of language translation using seq2seq ...
user avatar
1 vote
1 answer
15 views

Neural networks - calculating output manually if $x_1=x_2=0$ . Should this be easy to do?

This is a problem question I'm trying to make sure I understand from a past paper (with no solutions). The R output is below. ...
user avatar
2 votes
1 answer
42 views

MNIST with a TWIST, no labels given, only probabilities

Let's say we have basic MNIST dataset, and we have the same goal to predict the digit, BUT we're swapping all the labels by RED ...
user avatar
  • 131
-1 votes
0 answers
9 views

How can I train Mixture Density Neural Network? [closed]

I am learning Mixture Density Neural Network but it looks different from the usual neural network for regression problem. As far as I have understood from what I have read on the Internet, it gives ...
user avatar
  • 2,583
2 votes
1 answer
10 views

Is it possible to deal with datasets of graphs with different number of nodes in graph nural networks?

I'm dealing with a graph classification problem. In my dataset, each graph has som specific number of nodes. The number of nodes has a range of 1-1000 nodes. At inference time (after training), the ...
user avatar
  • 412
0 votes
0 answers
17 views

Is there a way of comparing/ evaluating a machine learning model with a recurrent model with a statistical significance test?

Comparing two different machine learning models (to assess if the difference between the mean performance is real or not using P-value and t-Statistic) is possible and strait forward in Python. ...
user avatar
  • 1
0 votes
1 answer
10 views

References for cross-validation implementations in Pytorch

I'm interested in good references on cross-validation implementations for feed-forward neural networks in pytorch from scratch. Thanks in advance.
user avatar
0 votes
0 answers
3 views

Comparison of multi-output binary classification tasks versus separate network for each of the binary tasks in terms of accuracy/AUC

If I have 5 separate binary classifiers that are using a pre-trained Inception V3 each separately would it provide less accurate results if I modify an inception V3 to create multi-output results as ...
user avatar
0 votes
0 answers
18 views

Advarsarial autoencoder loss function - Using MSE and BCE both

I came across this implementation of AAE on financial data to detect anomalies https://github.com/GitiHubi/deepAD/blob/master/KDD_2019_Lab.ipynb. In here for the VAE part of AAE, the author is using ...
user avatar
0 votes
0 answers
13 views

Is there a way of comparing/ evaluating a machine learning model with a recurrent model like LSTM with a statistical significance test? [closed]

Comparing two different machine learning models (to assess if the difference between mean performance is real or not using P-value and t-Statistic) is possible and strait forward in Python. Generally, ...
user avatar
  • 1
0 votes
0 answers
31 views

How is the feature interaction different in XGBoost vs. a fully connected neural network

When you know your features could interact with each other, will you choose XGBoost or NN-based models? My friend is training with an XGBoost, and he manually adds interacted features (X1 * X2) as new ...
user avatar
  • 31
0 votes
0 answers
23 views

RNN loss function spikes and slow decrease

I'm training a rnn model for classification (many to one) : ...
user avatar
  • 1
0 votes
0 answers
14 views

Text Extraction and Text recognition

Starting from text I'd like to be able to identify specific informations. Example : Input texts : "The invoice number is 18", "Inv : 75", "Inv N. : 84" Identified invoice ...
user avatar
  • 101
1 vote
0 answers
23 views

How to deal with unknown classes with a convolution neural network classifier?

I'm quite new into the DL and ML field. I'm training a CNN able to classify 3 different classes, however I would like in the testing phase to make the CNN able to not misclassify images that do not ...
user avatar
  • 11
0 votes
0 answers
10 views

Improve the perfomance of the deep learning model based on the train and validate loss curve [duplicate]

I have a deep learning model and the following is the loss on the train and validate data. The prediction for my model is not good. Do you know what I should do for my model to have a better results? ...
user avatar
  • 23
1 vote
1 answer
15 views

Do I need to normalize data before applying L1, L2 norm in ANN

I wish to train the ANN and use regularizers to avoid overfitting. I need some suggestions, is it mandatory to normalize the data before using L1, L2 regularizers. I would highly appreciate if you can ...
user avatar
  • 141
1 vote
1 answer
21 views

Concatenation or separate channels for a CNN

let's say I am classifying time series data from multiple channels in a biomedical setup (e.g. 12 lead ECG). I have been reading this paper on a CNN-based (ResNet) architecture for assesing the ...
user avatar
  • 111
0 votes
1 answer
7 views

Does the attention mechanism (in CNNs) bring additional parameters/weights to learn to the network?

The idea of the attention mechanism is based on using some weighted sum of the output of some layers in deep networks. I see the process in forward propagation, and it seems that the attention ...
user avatar
  • 189
0 votes
0 answers
16 views

Dependent variable standardization in neural networks

I am using a multilayer perceptron model to predict urban temperatures. I have standardized the independent variables before training the model. However, I have not standardized the dependent variable....
user avatar
0 votes
0 answers
46 views

How can I get the Binary Cross Entropy from the Cross Entropy function for GANs

I got the definition of log-likelihood by Goodfellow's Deep Learning book: \begin{equation} \label{eq:loglikelihood} \theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta). \end{...
user avatar
0 votes
0 answers
26 views

Statistical test when comparing oversampling to no oversampling on ANN

I use 70% of the dataset for training and 30% for testing. I use oversampling on the training dataset with an ANN. I use the test dataset on my ANN and look at the performance of oversampling against ...
user avatar
0 votes
0 answers
26 views

Can somebody help me understand the sentences in more readable expresions?

I was reading a paper about "bayesSimIG" and I have problem in understand the following paragraph.I have read it many times and did a lot of research for it and have understood what each ...
user avatar
  • 2,583
0 votes
0 answers
19 views

Neural Network $\delta^L$ is very often zero [closed]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it, I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
15 views

Neural Network not learning [duplicate]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
9 views

How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]

I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
user avatar
0 votes
0 answers
9 views

Using Inception and FID scores in training?

Is it possible to use the Inception and FID scores in the training of a deep image generation model, i.e. to maximize the scores in a loss function, albeit this is "cheating"? If so, has ...
user avatar
  • 1
0 votes
1 answer
21 views

Finding the position of the global optimum with Pytorch [closed]

I have a dataset with 22 parameters and I did a PolynomialFeatures = 2 to find the influence of interaction. This was then fitted to an Artificial Neural Network with the lowest loss being around 0.02....
user avatar
  • 21
0 votes
0 answers
11 views

Shrinkage / L1 regularization as a loss term versus a constraint (post-process step) with momentum optimizers

I have a complex model with very non-linear operations (divisions, exponentials, matrix inversions, square roots, Cholesky decompositions, etc...) for which I want to optimize the parameters. However, ...
user avatar
2 votes
1 answer
41 views

Should I join train and validation sets for final NN model training? If yes, when to stop training the final model?

Normally we divide our dataset into 3 sets: train set, validation set, test set. We use train set to find optimal parameters (weights and biases of NN) and validation set to find optimal NN ...
user avatar
1 vote
0 answers
10 views

is nrmse scale-dependent?

Im trying to evaluate my regression models using a normalised version of the RMSE, nrmse = rmse(y, y_pred)/rmse(y, y_mean) where y_mean is the array of the same len as y filled with the mean value of ...
user avatar
  • 11
1 vote
0 answers
13 views

Encoding Geolocation data

I am working on routing bus from one stop to another, for which Geolocation data inform of latitude and longitude is required. In addition to xy coordinates, distance matrix of locations is also ...
user avatar
0 votes
0 answers
17 views

Best way to approximate head point having only face keypoints

I'm using the BlazeFace model from TensorFlow which only has this few keypoints: I need those keypoints plus a head keypoint, like this one: My question is, which would be the best way to ...
user avatar
0 votes
0 answers
7 views

Validation Loss for my binary image classifier model is increasing. how to bring it down? [duplicate]

I am new to the domain of Deep learning and I have been trying to create a binary image classifier using a dataset which I created by myself. I am building the model from scratch. It is CNN model. ...
user avatar
  • 1
0 votes
1 answer
30 views

How is the extrapolation variant of the parity problem defined?

In the paper PonderNet: Learning to Ponder (Banino et al. 2021), the authors define the following "Parity" task: input vectors had 64 elements, of which a random number from 1 to 64 were ...
user avatar
1 vote
1 answer
13 views

In a tranformer, the same word can have different attention weights in different sentences?

I'm trying to understand the transformer architecture for NLP. The main issue is regarding the attention weights. The same word can have different attention weights in different sentences, right?
user avatar
  • 412
1 vote
0 answers
13 views

How to tell a conditional generative model is overfitting or not? And how to split training and validation set?

I'm now working on conditional generative models, but it confused me in the training process. My process: split training and validation set, and a held-out test set train the model and evaluate the ...
user avatar
0 votes
0 answers
13 views

How to properly mask MultiHeadAttention for sliding window time series data

I have data in the shape (batch, seq_len, features) that is a time series sliding window. In essence, I'm using the most recent ...
user avatar
0 votes
0 answers
19 views

and what if non-linear activation functions give better results than the linear ones?

I had a regression problem with small data set, I solved it with neural networks (MLP, ELM,..) As convention, I used a linear function for output layer, the results were not so good. I tried to change ...
user avatar
  • 153
2 votes
1 answer
31 views

Restrict output range of a neuron based on output of other neurons

I have a neural network with three output neurons $X_1$, $X_2$ and $X_3$ with output range in [-1, 1]. I have many training data split in 80:20 ratio between training & testing sets. While ...
user avatar
  • 363
1 vote
0 answers
25 views

Variance of the predicted temperature

I am predicting the electrical load and I also use the predicted temperature as one of the input feature. For example, I want to predict the electrical for tomorrow. I use the predicted temperature ...
user avatar
  • 23
0 votes
0 answers
17 views

Similar results between feedforward neural networks, recurrent neural network and LSTM for time series data - Is this standard?

Tl;dr: I have trained feedforward neural networks, recurrent neural networks and LSTM networks to predict behaviour of weather temperature. The results are almost all the same (see below). Is this ...
user avatar
  • 111
0 votes
0 answers
12 views

Can we compare rigorously the computing time to evaluate ReLU or other nonlinear smooth activations?

Can we say that, independently of the computer, computing relu and relu' is cheaper than computing f and f' for some other smooth non-linear activation (e.g. logistic, tanh)? If not, what are the ...
user avatar
0 votes
0 answers
27 views

How to calculate Cosine Similarity from Keras model?

I'm trying to make hybrid recommender system that recommends movies to users from Movielens dataset. Its Content part is based on Doc2Vec model from gensim library and its Collaborative Filtering part ...
user avatar
0 votes
0 answers
23 views

How to choose the best recommender system? What evaluation metrics to use?

I want to build a recommender system to suggest similar songs to continue a playlist (similar to what Spotify does by recommending similar songs at the end of a playlist). I want to build two models: ...
user avatar
  • 1
1 vote
0 answers
11 views

Backprop in Residual neural network?

I'm trying to build a Residual neural network with 2 layers , and I'm having difficultiy understanding what are the equations for the backprop for the following : $$ W_{2}x+tanh(W_{1}x+b_1)+b_2 $$ ...
user avatar
0 votes
0 answers
35 views

Why does Adam optimizer with gradient clipping perform better than simple Adam optimizer?

Since Adam optimizer uses the first and second moments of gradients to adapt the learning rate, what purpose does the gradient clipping serve when used with Adam optimizer or any adaptive learning ...
user avatar
  • 151
0 votes
0 answers
7 views

Customizing anomalies for different customers

I have built an LSTM autoencoder model to identify anomalies in time series wifi throughput data for over 100 customers. However, the definition of anomalies is very subjective. E.g. Customer A thinks ...
user avatar
0 votes
0 answers
14 views

How to report neural network results on test data set

I am working on a manuscript for which I have results from a simple neural network that I would like to report. I have both a training dataset and a test dataset. My metrics of interest include root ...
user avatar
  • 185

1
2 3 4 5
186