Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

0
votes
0answers
10 views

What is the encoding in VAEs?

The question is actually less broad than it sounds. I generally do understand how variational autoencoders work. From the encoding step we get four parts: mean $\mu$ standard deviation $\sigma$ ...
1
vote
0answers
12 views

Why using RMSE as loss function in logistic regression takes non convex form but doesn't in linear regression?

I am taking this deep learning course from Andrew NG. In the 3rd lecture of 2nd week of the first course, he mentions that we can use RMSE for logistic regression as well but it will take a nonconvex ...
0
votes
0answers
10 views

How is inference performed in the RNN “many-to-one” architecture?

I'm looking at the diagram for the "many-to-one" architecture here and it looks like in the training phase there would be weights trained across activations between timesteps $W_{aa}$, between inputs ...
0
votes
0answers
7 views

Variable parts of hidden layers in a network

my question is about a variable output or some parts of a net which can vary. For example a flag would direct which output (or some part of a net) is to choose. That means that I have different hidden ...
0
votes
0answers
11 views

What type of accuracy should I reported in research paper?

I have read some research papers on the classification task of deep learning, and now I am doing my own. After investigating some research paper which also provided the source code for reproducing ...
1
vote
0answers
12 views

Model with target as feature for benchmark

I was reading this article (https://towardsdatascience.com/predicting-the-popularity-of-instagram-posts-deeb7dc27a8f) about predicting the popularity of Instagram post using different techniques and ...
1
vote
1answer
15 views

training a nn with f1 as loss on keras doesn't work?

I have no problem to train my neural network with categorical_crossentropy as loss but when I do the same with f1, it just doesn't progress : Epoch 1/9 1029/1029 [==============================] - ...
1
vote
2answers
22 views

In CNN, how to map from fully connected layer to output image?

In CNN, where the last layers are fully connected, how to make pixel-wise prediction to output an image(binary matrix), if the number of neurons in the last layer is less than the size of the image?
0
votes
0answers
19 views

Using sigmoid in binary DNN output layer instead of softmax?

For a binary DNN, the output is $y_0 + y_1 = 1$ since they are the probability distribution, hence the sum must equate to 1. However, I've been told that $y_1$ is sufficient to represent the output of ...
2
votes
1answer
25 views

Can the temperature of neural network be used to calculate probability distribution?

Neural Networks have "temperature" - how much randomness should be added to the result. Would it be possible to use that temperature to get not just single number prediction, but the probability ...
0
votes
0answers
8 views

Predict Probability Distribution with Neural Network or Monte Carlo

Let's say we would like to predict price of Microsoft Stock. We have historical data and interested in predicting price distribution for future time t+1, like shown ...
0
votes
0answers
14 views

Recurrent Neural Network - Time Series prediction: How to decide on the sequence length and number of output neurons

Context I am currently working with Gated Recurrent Units for time series prediction. Anyhow, my question should apply to all kinds of such recurrent networks. In my data, I have timestamps of 15 ...
1
vote
0answers
22 views

Training Neural Network with Simulated Annealing

I am trying to train a simple neural network with simulated annealing. I have programmed a neural network with an input layer of 784 input nodes (28 x 28 pixels, I am using the MNIST database to train)...
1
vote
1answer
21 views

Enforcing Dirac delta-like Activations on a Neural Network

I am working on a custom neural network model including convolutional and dense layers. I intend to enforce outputs a certain dense layer to approximate a Dirac delta function (or any localized pulse)....
0
votes
1answer
11 views

Multi-task learning: weight selection for combining loss functions

I am training a system that combines two sub-systems: one for classification and another for reconstruction. Can anyone suggestion what are the common practice for weight selection for combining two ...
0
votes
0answers
16 views

What are other nonlinear transformation methods in machine learning except Neural Network activation functions?

One advantage of the MLP neural networks is the nonlinear transformation used on raw features. The popular ones used are the activation functions like Sigmoid, Tanh, ReLU, Leaky ReLU, etc. They are of ...
1
vote
1answer
28 views

Inputting playing card values to aneural network

I am trying to create a NN to play a card game wherein each state is represented by the hands of 4 players. Every round, the hand of each player is decreased by 1 (discarded). Each player starts with ...
0
votes
0answers
8 views

Pytorch logging: Native tensorboard support v/s TensorboardX

PyTorch recently released v 1.1.0, which has native support for Tensorboard. How does this compare with TensorboardX? I thought it would be good to list the pros and cons here. I am new to PyTorch ...
1
vote
0answers
8 views

Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
0
votes
0answers
4 views

LSTM/RNN history-based prediction by using Keras backend?

For my experiment, I have a formatted csv file with 1440 columns like following: ...
0
votes
0answers
5 views

Estimating a changing transit time between inputs and output

I work with a chemical process in which there is a time lag between the inputs (raw material quality and cooking parameters) and the output (final product quality). The problem is that the time lag ...
0
votes
1answer
14 views

Training Perceptrons with Backprop

Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule? is it ...
0
votes
0answers
17 views

why the validation accuracy of deep CNN is high but not stable

I'm training a CNN on some vibration data, and I get some somewhat strange results. I found the validation accuracy is unstable. In some epoch, the val_acc may be 90%+, but in the next epoch, it may ...
1
vote
1answer
20 views

Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
0
votes
0answers
24 views

Sensitivity Analysis with categorical predictive variables in R

I am doing a project where I have to predict the Sales Units in fashion and intend to run a Random Forest, Neural Networks and Support Vector Machine models. However, my predictive variables are all ...
0
votes
0answers
23 views

Is Stochastic Gradient Descent sensitive to training permutation?

I've recently read that SGD (Stochastic Gradient Descent) is one of the most popular techniques for training Machine Learning algorithms, including DNNs (deep neural networks). However, my ...
0
votes
1answer
33 views

What are general practises used to divide the data into training / dev and test set?

Example: I have am building a dog vs cat classifier and I have collected data from 15 countries. Europe: 1. UK 2. France 3. Germany 4. Italy 5. Finland Asia: 1. India 2. China 3. Japan 4. Russia 5. ...
0
votes
1answer
19 views

Should we normalize target data as well as input data?

I consider myself an intermediate practitioner of neural networks. I've been asked to teach a few of my colleagues some of what I know. Some of my practices may be a bit idiosyncratic, because I study ...
1
vote
0answers
24 views

Why we always get different accuracy for a different number of training our model? [duplicate]

As the question says for example if I train my neural network (with 2 layers) model the first time it gives me a score $A \in \mathbb{R}$ and when I train the same model again it gives a different ...
0
votes
0answers
28 views

What to choose?ML project or an Internship? [on hold]

This is slightly off topic but pretty serious for me. I am an undergraduate student in CSE, 3rd year. I am confused in whether to do internship or to make my own project in Machine Learning in my ...
1
vote
1answer
22 views

Reproducible numbers in Keras/TensorFlow

Every time I run a Keras/TensorFlow code gives different results. Can someone suggest how to get reproducible numbers?
0
votes
0answers
13 views

Mutual Information between layer's activation and class label

I want to calculate Information gain of particular layer's activation with respect to class label, which is something quite similar to, ...
0
votes
0answers
11 views

What does a word embedding's dimension signify?

I'm currently studying NLP and had a question regarding word embeddings. My understanding of a word embedding is that it is, simply put, a modular way of expressing words and phrases as vector ...
0
votes
0answers
3 views

How does short-term dependency improve performance for NLP models?

I was reading a paper titled Sequence to Sequence Learning with Neural Networks (Sutskevers, Vinyals, and Le - 2014 NIPS) and had a question regarding the concepts of "short-term dependency" and "long-...
0
votes
1answer
8 views

Setting bias of output layer for imbalanced datasets

From a blog post from Andrej Karpathy on training neural networks: Initialize the final layer weights correctly. E.g. if you are regressing some values that have a mean of 50 then initialize the ...
1
vote
0answers
26 views

What is state of the art in gradient free neural network learning, esp. for images?

I've been recently looking into gradient free learning of neural networks. However, most of the techniques I've found seem to be only applied to toy problems, which I assume means they're infeasible ...
0
votes
0answers
27 views

Calculating the number of neurons and the number of hidden layers for a neural network MATHEMATICALLY

I have a fair idea that a lot of research has been done and is still underway to explore the science behind the black art of a neural network (NN) architecture, i.e., accurately calculating the number ...
1
vote
0answers
15 views

What could cause a flat loss function to suddenly decrease in a u-net used for denoising?

So I am trying to understand U-Nets better, and I built a very shallow U-Net and trained it to denoise MNIST images (training set is 90% of the whole dataset). The loss function evolution I obtained ...
3
votes
1answer
67 views

Is it valid to have all zeroes in a One-Hot Encoded categorical feature?

I'm building an MLP classification model and one of my features is the name of certain products. These names can be anything and in theory there could be an infinite number of different names in the ...
1
vote
0answers
31 views

Cross validation for time series prediction: How to choose the best model from different neural networks?

I want to choose the best model from a list of neural network models. My problem is a multivariate time series forecast (regression) problem, in which I forecast a parameter using other parameters, up ...
0
votes
1answer
22 views

Back propagation is done with each batch in a convolutional net, but is it also done with the validation set?

It's my understanding that the weights are updated in a convolutional neural network with each evaluation of a batch. But when the training data has been processed and it comes to predicting ...
0
votes
0answers
13 views

How do I implement masking in TensorFlow eager?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
0
votes
0answers
4 views

what is the difference between sklearn's train_test_split and keras load_data()?

im experimenting on autokeras, while doing so i came across something like (x_train, y_train), (x_test, y_test) = mnist.load_data(), is this different from sklearn.model_selection....
0
votes
0answers
21 views

How to add new features to already trained model without training again on whole dataset?

Suppose, we have following features on which a classification model (Neural Network) is trained to predict whether a customer will buy Milk or not (0 :Will not buy, 1:Will buy) each week(n): ...
0
votes
0answers
14 views

Gradient clipping just before averaging

A typical way of implementing mini batch learning is by calculating the gradients of every element within the mini batch and then average all of these element's gradients to come up with the final ...
0
votes
1answer
38 views

A model (neural network) for sets of arbitrary length [closed]

I've been searching for a model that is close to RNN (is well suited for investigating sets of arbitrary lengths) but is insensitive to order. I'm aware of bidirectional RNNs. I've also found a 'bag ...
1
vote
0answers
23 views
+50

Where does the prior distribution $p(z)$ for adversarial autoencoders come from?

I am trying to understand how an adversarial autoencoder works. The discriminator takes as an input the aggregated posterior $q(z)$ generated from the decoder and matches this against the prior ...
0
votes
0answers
15 views

Why does 4-gram work better than trigram or bigram or unigram in my experiments?

In a binary classification task, I used Logistic regression, decision tree and Adaboost with decision tree (max_depth=1). For each of the machine learning task, I used GridSearchCV to choose the ...
0
votes
0answers
42 views

How do you define the sensitivity of a neural network? [closed]

What are the sensitivities of a neural network with a sigmoid output node, and two relu hidden nodes? In this context, by "sensitivity" I mean he sensitivity of the function with respect to that ...
0
votes
1answer
24 views

What is meant by Low-Order combination of features?

I came across a Machine Learning paper that talks about input with low-order combination of features. A statement says: The initial feature is used as the input of the model, and the non-linear ...