Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

0
votes
0answers
8 views

Inputting playing card values to aneural network

I am trying to create a NN to play a card game wherein each state is represented by the hands of 4 players. Every round, the hand of each player is decreased by 1 (discarded). Each player starts with ...
0
votes
0answers
3 views

Pytorch logging: Native tensorboard support v/s TensorboardX

PyTorch recently released v 1.1.0, which has native support for Tensorboard. How does this compare with TensorboardX? I thought it would be good to list the pros and cons here. I am new to PyTorch ...
1
vote
0answers
5 views

Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
0
votes
0answers
3 views

LSTM/RNN history-based prediction by using Keras backend?

For my experiment, I have a formatted csv file with 1440 columns like following: ...
0
votes
0answers
4 views

Estimating a changing transit time between inputs and output

I work with a chemical process in which there is a time lag between the inputs (raw material quality and cooking parameters) and the output (final product quality). The problem is that the time lag ...
0
votes
1answer
13 views

Training Perceptrons with Backprop

Is it possible to train a simple perceptron with a threshold activation function such as this one: https://en.wikipedia.org/wiki/Perceptron with Backpropagation instead of the perceptron rule? is it ...
0
votes
0answers
14 views

why the validation accuracy of deep CNN is high but not stable

I'm training a CNN on some vibration data, and I get some somewhat strange results. I found the validation accuracy is unstable. In some epoch, the val_acc may be 90%+, but in the next epoch, it may ...
1
vote
1answer
12 views

Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
0
votes
0answers
14 views

Sensitivity Analysis with categorical predictive variables in R

I am doing a project where I have to predict the Sales Units in fashion and intend to run a Random Forest, Neural Networks and Support Vector Machine models. However, my predictive variables are all ...
0
votes
0answers
21 views

Is Stochastic Gradient Descent sensitive to training permutation?

I've recently read that SGD (Stochastic Gradient Descent) is one of the most popular techniques for training Machine Learning algorithms, including DNNs (deep neural networks). However, my ...
0
votes
1answer
28 views

What are general practises used to divide the data into training / dev and test set?

Example: I have am building a dog vs cat classifier and I have collected data from 15 countries. Europe: 1. UK 2. France 3. Germany 4. Italy 5. Finland Asia: 1. India 2. China 3. Japan 4. Russia 5. ...
0
votes
1answer
17 views

Should we normalize target data as well as input data?

I consider myself an intermediate practitioner of neural networks. I've been asked to teach a few of my colleagues some of what I know. Some of my practices may be a bit idiosyncratic, because I study ...
1
vote
0answers
23 views

Why we always get different accuracy for a different number of training our model? [duplicate]

As the question says for example if I train my neural network (with 2 layers) model the first time it gives me a score $A \in \mathbb{R}$ and when I train the same model again it gives a different ...
0
votes
0answers
25 views

What to choose?ML project or an Internship? [on hold]

This is slightly off topic but pretty serious for me. I am an undergraduate student in CSE, 3rd year. I am confused in whether to do internship or to make my own project in Machine Learning in my ...
1
vote
1answer
22 views

Reproducible numbers in Keras/TensorFlow

Every time I run a Keras/TensorFlow code gives different results. Can someone suggest how to get reproducible numbers?
0
votes
0answers
11 views

Mutual Information between layer's activation and class label

I want to calculate Information gain of particular layer's activation with respect to class label, which is something quite similar to, ...
0
votes
0answers
10 views

What does a word embedding's dimension signify?

I'm currently studying NLP and had a question regarding word embeddings. My understanding of a word embedding is that it is, simply put, a modular way of expressing words and phrases as vector ...
0
votes
0answers
3 views

How does short-term dependency improve performance for NLP models?

I was reading a paper titled Sequence to Sequence Learning with Neural Networks (Sutskevers, Vinyals, and Le - 2014 NIPS) and had a question regarding the concepts of "short-term dependency" and "long-...
0
votes
1answer
6 views

Setting bias of output layer for imbalanced datasets

From a blog post from Andrej Karpathy on training neural networks: Initialize the final layer weights correctly. E.g. if you are regressing some values that have a mean of 50 then initialize the ...
1
vote
0answers
26 views

What is state of the art in gradient free neural network learning, esp. for images?

I've been recently looking into gradient free learning of neural networks. However, most of the techniques I've found seem to be only applied to toy problems, which I assume means they're infeasible ...
0
votes
0answers
26 views

Calculating the number of neurons and the number of hidden layers for a neural network MATHEMATICALLY

I have a fair idea that a lot of research has been done and is still underway to explore the science behind the black art of a neural network (NN) architecture, i.e., accurately calculating the number ...
1
vote
0answers
15 views

What could cause a flat loss function to suddenly decrease in a u-net used for denoising?

So I am trying to understand U-Nets better, and I built a very shallow U-Net and trained it to denoise MNIST images (training set is 90% of the whole dataset). The loss function evolution I obtained ...
3
votes
1answer
48 views
+100

Is it valid to have all zeroes in a One-Hot Encoded categorical feature?

I'm building an MLP classification model and one of my features is the name of certain products. These names can be anything and in theory there could be an infinite number of different names in the ...
1
vote
0answers
28 views

Cross validation for time series prediction: How to choose the best model from different neural networks?

I want to choose the best model from a list of neural network models. My problem is a multivariate time series forecast (regression) problem, in which I forecast a parameter using other parameters, up ...
0
votes
1answer
22 views

Back propagation is done with each batch in a convolutional net, but is it also done with the validation set?

It's my understanding that the weights are updated in a convolutional neural network with each evaluation of a batch. But when the training data has been processed and it comes to predicting ...
0
votes
0answers
12 views

How do I implement masking in TensorFlow eager?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
0
votes
0answers
3 views

what is the difference between sklearn's train_test_split and keras load_data()?

im experimenting on autokeras, while doing so i came across something like (x_train, y_train), (x_test, y_test) = mnist.load_data(), is this different from sklearn.model_selection....
0
votes
0answers
19 views

How to add new features to already trained model without training again on whole dataset?

Suppose, we have following features on which a classification model (Neural Network) is trained to predict whether a customer will buy Milk or not (0 :Will not buy, 1:Will buy) each week(n): ...
0
votes
0answers
14 views

Gradient clipping just before averaging

A typical way of implementing mini batch learning is by calculating the gradients of every element within the mini batch and then average all of these element's gradients to come up with the final ...
0
votes
1answer
37 views

A model (neural network) for sets of arbitrary length [on hold]

I've been searching for a model that is close to RNN (is well suited for investigating sets of arbitrary lengths) but is insensitive to order. I'm aware of bidirectional RNNs. I've also found a 'bag ...
1
vote
0answers
17 views
+50

Where does the prior distribution $p(z)$ for adversarial autoencoders come from?

I am trying to understand how an adversarial autoencoder works. The discriminator takes as an input the aggregated posterior $q(z)$ generated from the decoder and matches this against the prior ...
0
votes
0answers
14 views

Why does 4-gram work better than trigram or bigram or unigram in my experiments?

In a binary classification task, I used Logistic regression, decision tree and Adaboost with decision tree (max_depth=1). For each of the machine learning task, I used GridSearchCV to choose the ...
0
votes
0answers
42 views

How do you define the sensitivity of a neural network? [closed]

What are the sensitivities of a neural network with a sigmoid output node, and two relu hidden nodes? In this context, by "sensitivity" I mean he sensitivity of the function with respect to that ...
0
votes
1answer
23 views

What is meant by Low-Order combination of features?

I came across a Machine Learning paper that talks about input with low-order combination of features. A statement says: The initial feature is used as the input of the model, and the non-linear ...
0
votes
0answers
11 views

Extracting likelihoods from generative model

I am looking for papers dealing with the extraction of explicit descriptions of probability distributions from a generative model. My use case is the following: I trained a GAN to generate samples ...
0
votes
0answers
21 views

Which algorithm should I use to predict the winner/loser of a competition, among 5 competitors?

I hope I posted in the correct session. I need to solve this "simple" problem. PROBLEM EXPLANATION I need to predict who is more likely to win a car race, among 5 drivers. I have a database that ...
0
votes
1answer
44 views

Should I Choose the best model based on test error or validation error?

I divided my dataset to training, validation and test sets. Then trained multiple forecasting models on the training dataset. now I have 3 errors for each model: Training error Validation error Test ...
0
votes
0answers
20 views

BERT Classification fine tuning for Q & A

I want to fine-tune BERT for Q & A in a different way than the SQuAD mission: I have pairs of (question, answer) Part of them are the correct answer (Label - 1) Part of them are the incorrect ...
0
votes
1answer
16 views

Variables preparation for MLP regression

I'm trying to create a regression model with MLP to predict a continuous variable, that is the income of a movie. My set of regressors is composed by around 15 binary variables (I've used one-hot-...
0
votes
1answer
25 views

Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?

Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
0
votes
1answer
30 views

Replacing CNNs with Random Forests

Suppose I have a sequence like "ADTGESW". Each character in this sequence can attain a number of possible values, let's say 10. I can then one-hot encode this sequence and obtain a matrix with shape ...
0
votes
1answer
19 views

Prediction of a continuous variable

I'm trying to create a model to predict a continuous variable, that is the revenue of a movie, given many predictors, such as its budget, the length of the film, the genre... I'm planning to use MLP, ...
-1
votes
0answers
11 views

How to know best parameters in DeepSpeech? [duplicate]

I am trying to train a NN with DeepSpeech in Spanish. I am using a 24 hour corpus, which I think would be large enough, but the transcriptions I am getting are completely useless. I am using 33 epoch, ...
0
votes
0answers
16 views

Literature recommendation for convolutional neural nets

I am looking for a good book or an article concearning convolutional neural nets, especially their architecture. I like the http://deeplearningbook.org but it does not provide any information on the ...
0
votes
0answers
41 views

What does E_n mean

I came across the following symbol in this paper: I am a little bit confused by the symbol . Does it simply mean population average?
1
vote
1answer
28 views

Testing methodology

I recently built a simple feed forward NN to predict daily demand (48 output neutrons, representing half hours) based upon 32 input features. I tested the performance by firstly doing 10 fold cross ...
0
votes
1answer
18 views

How best to combine object detection and tracking

I am trying to make a computer vision system which will be able to detect and track objects of interest. This will require (1) detection functionality to notice the object when it appears (2) ...
0
votes
0answers
43 views

Which neural network models can learn local correlation between features better? [closed]

Following this question and this paper , let's say I have time-series data on 3 physical parameters which are in 3 matrices form R,G,B and all are in the same size like N×K and I combined them and ...
1
vote
0answers
23 views

Is stationarity a requirement when using neural networks for time series forecasting?

I'm getting conflicting information on whether stationarity is a requirement when using neural networks for time series forecasting: In this lecture, the speaker says it isn't a requirement. In this ...
0
votes
1answer
36 views

Which machine learning or deep learning model to use / make [closed]

I wanted to use machine learning / deep learning to predict profit of a , say, a shop using satellite imagery and previous data and I don't know what to use. I want to use close satellite imagery to ...