Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [neural-networks]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

0
votes
1answer
9 views

Is there a way to check the full learnt function by Neural Network, not only the weights?

The training is mostly learning about the wights.But what about the full function learnt by NN? In typical deep learning framework, is there a way to example the function learnt? For example: ...
0
votes
0answers
12 views

Why Transformer use little activations?

I am confused about the activation functions used in the famous NMT model Transformer. In other classical networks such as ResNet and LSTM, the activation functions are used after every linear ...
0
votes
1answer
12 views

Do I include the validation set in final training?

For optimizing an unsupervised neural network with 1 hidden layer, I use the training set for training and the validation set for optimizing the number of neurons in the hidden layer (for example by ...
0
votes
1answer
15 views

Reinforcement Learning: Actor Critic - Why is weight sharing possible?

I was looking at Open Ai's actor-critic code and noticed that some of the neural network's weights are shared ...
0
votes
1answer
24 views

Why Energy in Restricted Boltzmann Machine?

In Restricted Boltzmann Machine (RBM), we define the energy function $E(\mathbf{v}, \mathbf{h}; \, \mathbf{W}, \mathbf{a}, \mathbf{b})$. $\mathbf{v}$ is visible unit $\mathbf{h}$ is hidden unit $\...
0
votes
0answers
35 views

How can RMSE be compared between a regression model and a neural network model?

In the calculation of RMSE, linear regression uses degrees of freedom(n-p) as divisor and neural network(feed-forward in my case) uses the total data number(does it have degrees of freedom as well?). ...
0
votes
0answers
15 views

Implicit regularization in Linear models

Regarding Linear Neural Networks models with unique finite root loss function, without an explicit regularization, I am struggling to prove that in the case of overparmeterized models (i.e. $N<d$), ...
0
votes
1answer
23 views

forget_bias interpretation in tensorflow

In Basic LSTM cell of tensorflow there is an argument named forget_bias. From the documentation of ...
0
votes
1answer
23 views

Generative model to generate hidden activations coming from a previously trained hidden layer

I need to train a generative model to generate vectors which resemble the activations of a particular hidden layer of a neural network which has been previously trained. In particular, the hidden ...
0
votes
0answers
12 views

How to choose the best algorithm to fit neural model [on hold]

Im beginner in machine learning .. I have a problem of classification and i want to fit a neural network model with R and using the function Neuralnet(). but i don't know what is the best algorithm to ...
1
vote
1answer
29 views

What kind of impact do autoencoders have on final model performance when compared to models trained only on supervised data? [on hold]

For example, say we have two datasets, a labeled set (I will call it df_labeled) of nrows=200k and an unlabeled dataset (df_unlabeled) of nrows=800k and we want to build a binary classifier. I clearly,...
0
votes
0answers
25 views

What does the error of the neural network model mean? [on hold]

I m fiting a neural network model using R and with the library(neuralnet) but i found the error of the neural network model 500.222 that is not logical at all .. I got this Error when i wrote this ...
2
votes
2answers
34 views

How does training affect the norm of weight matrices?

I have a neural network $F(W,x): \mathbb{R}^d \rightarrow \mathbb{R}^k$ with $L$ layers, $m$ neurons per layer, ReLu activation, softmax on the last layer and $n$ datapoint. My loss function is the ...
0
votes
0answers
7 views

Calculating the gradient used by the Jacobian-based Dataset Augmentation in Keras [migrated]

I am trying to understand my mistake when calculating the sign of the gradient used by the jacobian-based dataset augmentation published in https://arxiv.org/abs/1602.02697 " the sign of the ...
0
votes
0answers
18 views

Can a trained neural network recognize rotating characters? [duplicate]

Suppose I have a trained neural network that can recognize, for example numbers from 1 to 10, the size of the picture $28 \times 28$. I made the rotation of these pictures by 90 degrees. Does now ...
1
vote
0answers
28 views

Training a neural network with partial labeling

I want to train a neural network that is part of a multi-armed bandit problem. For each data sample, I have some features representing the context of the sample and there are x neurons in the output ...
0
votes
1answer
16 views

image caption generator

I see two models of image caption generator online: In the above model, the first LSTM cell of decoder takes the entire image as an input. In the above model, all the LSTM cells of the decoder take ...
2
votes
1answer
17 views

Simple question related to an ANN diagram showing all of the matrices' dimensions: inputs, weights and output

I've been reading a few Neural Networks articles for the past week and one thing that I am still trying to grasp is the dimensioning of the matrices on an ANN training. I have created a diagram (based ...
2
votes
0answers
19 views

Validation ROC AUC not improving with validation cross-entropy loss?

I am training a neural network that is doing binary image classification on several thousand images. I am running 5 fold cross validation (train on 4, validate on 1) with cross entropy (CE) loss. I am ...
0
votes
1answer
18 views

Discard features with small variance, how to do in practice? [duplicate]

I'm training a neural network for regression. The input vectors consist of $92$ different features, I want to discard features with small variance (standard deviation). There are two ways that came ...
0
votes
0answers
37 views

LSTMs and Opening/Closing Brackets

I'm training a character-level LSTM to generate molecules using the SMILES system. Each molecule is represented as a string of characters, looking something like this: Cn1c(Nc2c(Cl)ccc(CNC(=O)C(C)(C)...
1
vote
0answers
11 views

How to incorporate uncertainty and noise information in training and prediction of neural networks?

I am trying to use RNNs to perform state estimation on noisy sensor data. The readings are from a GPS dataset and it provides $[longitude, latitude, n_{satellites}]$. The last column, which is the ...
0
votes
2answers
33 views

What are the predictor variables in a neural network?

In a linear regression model, the predictor or independent (random) variables or regressors are often denoted by $X$. The related Wikipedia article does, IMHO, a good job at introducing linear ...
1
vote
1answer
45 views

How to to get normal distributed neural network output [closed]

I am trying to build a neural network that predicts a pair of latitude / longitude coordinates following a previous pair of latitude and longitude (highly simplified). The latitudes and longitudes ...
1
vote
1answer
44 views

Should I discard a feature whose max - min is smaller than 1e-5?

I'm training a neural network for regression. The input vector consists of $140$ entries. For each feature vector entry, I calculate both min and ...
0
votes
0answers
23 views

Unable to implement multiple hidden layers in an RNN (PyTorch) [closed]

My Pytorch RNN for name classification does not allow me to choose multiple hidden layers. If I choose more than 1 layer I get the following error message: Traceback (most recent call last): File "...
0
votes
1answer
20 views

Residual Network: Is small deviation of layer response good? What's the point of resnet?

I have 2 questions... In the paper Deep Residual Learning for Image Recognition, it says We show by experiments (Fig. 7) that the learned residual functions in general have small responses, ...
0
votes
1answer
27 views

Is splitting data randomly into train, validation and test sets a bad idea?

In Splitting into train, dev and test sets it is recommended that It is important to choose the dev and test sets from the same distribution and it must be taken randomly from all the data. I have ...
1
vote
1answer
42 views

Hard attention loss function

I am referring to paper: Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (page 4). I wished to know, why we look to maximize the lower bound of the log likelihood ...
0
votes
0answers
8 views

How does network structure (model complexity) affects covergence speed?

I trained Bi-GRU and HAN (Hierarchical Attention Networks) on my own datasets, and found HAN converges faster than Bi-GRU, within less number of epochs. What would be the reason for this? I guess ...
0
votes
1answer
32 views

What does a “similar” dataset mean in the context of fine tuning a CNN?

In https://arxiv.org/pdf/1809.09529.pdf it is said If the new dataset is similar to original dataset, we expect higher-level features in the CNN to be relevant to this dataset. Thus, it is ...
0
votes
0answers
19 views

Difference of network between testing and training on the same dataset (No training and testing)

I was training and dense net model on emotion recognition on the sewa dataset. Therefore, at the end I have 2 outputs. One for arousal and the other for valence (These dimensions for emotions). So I ...
0
votes
1answer
35 views

Training error not decreasing on the training set

I cannot make my neural network - MLP with 1 hidden layer fit the training data perfectly. Here is the data: ...
1
vote
1answer
17 views

On masked multi-head attention and layer normalization in transformer model

I came to read Attention is All you Need by Vaswani. There two questions came up to me: 1. How is it possible to mask out illegal connections in decoder multi-head attention? It says by setting ...
0
votes
0answers
9 views

What is the best input for denoise autoencoder for sound/audio data?

I am currently trying to build an autoencoder to de-noise audio data. However I have not found any good articles explaining about the input to the autoencoder, i.e. feature vector. As in speech ...
0
votes
0answers
8 views

Reuse hidden layer in another Tensorflow model [closed]

I have a Tensorflow models consisting on a LSTM layer connected to a fully connected layer. This model has only one input. After the model is trained I want to use its weights as initial weights to a ...
1
vote
0answers
54 views

Hottest news on 'why does Deep Learning work so well' [closed]

At the beginning of last year I was trying to study some papers which were tackling the question "Why does deep learning work so well", but I had to stop due to overwhelming work problems. I'd love ...
0
votes
0answers
23 views

How sensitive are Neural Networks to weight changes?

Let's consider the space of feedforward neural networks with a given structure: $L$ layers, $m$ neurones per layer, ReLu activation, input dimension $d$, output dimension $k$. Which means I'm ...
1
vote
0answers
23 views

Could machine learning be used to select best parameter for more than one loop optimization?

I do not know if i can ask this question here or not. But I really need it. I'm very new to ML/DL/NN field. I have seen many articles tackling the problem of selection of the best parameter for loop ...
0
votes
1answer
12 views

How does DQN parameter updates work in simulation?

I've already read almost every Questions-answers and material related to DQN, deep reinforcement learning, but I'm struggling to start working on simulation. First of all, I'm trying to code using ...
0
votes
2answers
50 views

Regression or classification in neural networks

Given a simple data set to train with neural networks where i.e.: wine quality is the categorical output and measurements of acidity, sugar, etc. are the numerical inputs. The output can be written ...
0
votes
1answer
15 views

Data augmentation methods for Raman Spectra

I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try? I have found this paper which suggests ...
0
votes
1answer
39 views

When does my unsupervised autoencoder start to overfit?

I am working on anomaly detection using an autoencoder neural network with $1$ hidden layer. This is an unsupervised setting, as I do not have previous examples of anomalies. The input data has ...
0
votes
0answers
5 views

In which applications bayesian network are more promising to capture the uncertainities and understanding?

I know, there are lots of question on difference between NN and bayesian network, What i am specifically talking about modelling and uncertainty capturing? As we know, bayesian ML methods are very ...
1
vote
1answer
26 views

Strange batch loss in keras

Im training a Bidirectional RNN with keras.losses.MSE and have my dataset shuffled before training. I manually split it into validation and train data. However when ...
0
votes
0answers
17 views

Gradient descent does not converge to due noise in the data

For learning purposes I am trying to get a neural network to learn a fairly simple function (e.g. x => sin(x) + rand() * 0.01 but more complicated). I can ...
1
vote
0answers
16 views

Reverse twenty questions

So given a typology, we can spontaneously generate a series of dichotomies that identifies types. That's what the game twenty questions does. It creates twenty dichotomies based on a neural net it has ...
0
votes
1answer
32 views

Why is my multi-layer (with identity activation) neural net converging petter than single layer perceptron?

Before I ask my actual question, I just want to verify that the following is correct: A multi-layer neural network with an identity activation function is equivalent to a single layer (also with ...
0
votes
1answer
22 views

Classifying XOR grid with simple NN, but with more points on the grid

I am getting started with very simple Neural Networks/Multilayer Perceptrons. I successfully classified the XOR problem, but I wanted to explore so I created a grid such as . I used Tensorflow code ...
1
vote
1answer
107 views

Can t-SNE help feature selection?

I'm training a fully connected feed forward neural network for regression. Given one training example $(x_i, y_i)$, I need to convert the raw representation $x_i$ into an invariant representation $...