Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Why does the prediction accuracy decline when the max. num. of iterations is increased for ANN (MLPR)?

I use a gridsearch to define the optimal parameterization for a MLP regression task with python's scikit-learn. I am training a single numerical target from 15 predictors whilst varying: Activation ...
0
votes
0answers
2 views

Significance coefficients in logistic regression derived from Keras deep network

I have two independent variables (x1,x2), which I use to predict y (binary). With a logistic regression, I could just use the standard errors of each estimated coefficient to test them for ...
0
votes
1answer
12 views

What are the advantages of combining BiLSTM and CRF?

BiLSTM-CRF is a common model for sequence tagging (POS tagging, NER, ect.). What are the advantages of combining BiLSTM and CRF? What is the role of each one of the parts in this combination?
0
votes
0answers
18 views

Four possible actions instead of two … How is it possible?

I am interested in practicing the OpenAI gym exercises. I want to resolve Breakout-v0, but I face a little problem. Let me explain a bit more. Here is a little ...
0
votes
1answer
18 views

How to implement Batch Norm to Deep learning Neural Networks?

I'm studying at coursea.com Neural Networks with deep learning course. I have a problem with implementing A Batch Norm to Mini-Batch Gradient descent. More accurately, in gamma and beta hyper-...
0
votes
0answers
4 views

Random forecast generation for input into deterministic model

I will try not to ask to open ended of a question, but I am looking for material on how to generate more forecasts from a single forecast to input into a deterministic model. I am looking at some of ...
0
votes
0answers
8 views

Removing the effect of Time series X on time series Y, when their relation is unknown

I am working on a dataset of 6 years measurements of a water quality parameter called 'chla' ( parameter 'X') measured by a sensor for each year from May to October. The parameter has its own trend ...
5
votes
1answer
835 views

Is a neural network consisting of a single softmax classification layer only a linear classifier?

Since the softmax function is a generalization of the logistic function it is continuous and non-linear. So the output of the softmax layer is: softmax( weight_matrix * input_activation) ...
0
votes
0answers
9 views

best neural network for data with both time and space patterns

I am struggling to choose the correct neural network type for my problem set. It encodes forex data that has relationships in both time and space. For example difference of price close compared to a ...
0
votes
3answers
320 views

How to quantitatively determine when to stop training ANN

I've implemented an artificial recurrent neural network and want to start training it on a variety of tasks. I've extensive searching online and haven't found a satisfactory answer of how the ...
1
vote
2answers
8 views

Whats the horizontal and vertical axis denoting in the below SGD contour

What is that oscillation in the y axis in gradient descent contour.
0
votes
0answers
6 views

How to embed the preceding knowledge in the training?

I am trying to train a neural network that takes X0 and X1 as input, predict A as output. Here is part of training set. ...
0
votes
0answers
15 views

Why am I getting a good accuracy even with an overfitted CNN model?

I got a bad fit for my model and no convergence happened during the time of training. However, I still got an accuracy of 97%. Then, I trained my other model and got an accuracy of 91%. However the ...
0
votes
1answer
6 views

The best epoch has a higher valdation score compared to training score

Model : 3D CNN regularized with autoencoder Data : 3D MRI image data, 804 Training Set(Data Augmentation applied, 78 -> 804), 12 Valdiation Set(no aug), 12 Test Set(no aug). In general the epoch of ...
1
vote
0answers
8 views

Normalization functions in RNN LSTM

I've read somewhere that the tanh function was introduced in order to combat the problem of vanishing exploding gradient. However not many sources explain why ...
0
votes
0answers
8 views

Hypothesis Test for how quickly a variable drops off (Neural Network Training)

I have a series of Neural Networks being tested against each other, from which I can calculate the cost on each training example or averaged over some number of examples. Graphing this cost per batch ...
12
votes
3answers
8k views

Hidden Markov Model vs Recurrent Neural Network

Which sequential input problems are best suited for each? Does input dimensionality determine which is a better match? Are problems which require "longer memory" better suited for an LSTM RNN, while ...
3
votes
0answers
12 views

Extreme Image Noise Removal

I've been trying to solve a noise removal (from images) problem using deep learning and I've tried a lot of the newer architectures for noise removal including FFDNet, NLRN and MWCNN. The problem is, ...
0
votes
2answers
32 views

Why do earlier hidden layers learn slower?

I'm reading chapter 5 of Nielsen's textbook about vanishing gradients. He states: In at least some deep neural networks, the gradient tends to get smaller as we move backward through the hidden ...
1
vote
1answer
8 views

What are the probabilities in the embedding layer of a Word2Vec?

I am trying to understand how a Word2Vec is being trained. I understand that it can be trained using a CBOW and SkipGram. I am however lost as to what the probabilities are in the embedding layer. ...
2
votes
1answer
12 views

What is the dimensionality of the cost function with this specific ANN structure?

I have the following ANN architecture, the neuron is a sigmoid neuron: Where the weight and parameter matricies are given by: $$ \begin{vmatrix} & x1& & x2& & x3& \end{...
1
vote
2answers
201 views

Normal Neural Network for Image 100X100?

Thank you in advance for any help at all. So, I have created a neural network using back propagation and sigmoid function. It seems to work for XOR and images with size of 28X28. However, When I ...
2
votes
1answer
22 views

Seeking Suggestion on Image Dataset for Logistic Regression

The problem is based on binary classification (target and interference). I have an image dataset for which I can not use pixel intensities. I can use pixel coordinates only. Nevertheless, CNN will ...
1
vote
1answer
20 views

How does Stochastic Gradient Descent with momentum distinguish between local minima and global minima?

I have several questions regarding this. How does SGD momentum know to converge at global minina and skip over local minima? I read that "SGD momentum goes past the minima (due to its velocity build ...
0
votes
0answers
8 views

Regression with CNN Loss won't pass certain values

I'm working on a regression problem which since my dataset is images i prefered to use widely used CNNs.In general,i have to predict a single value between 0 and 500 for a 100x100 grid based on the ...
1
vote
1answer
13 views

Comparing Neural Network to ARMA

I used a neural network tool in MATLAB to predict data, and it gave it's accuracy as MSE and an R-value. I used the econometricModeler tool in MATLAB to predict data using ARMA. It gave it's accuracy ...
2
votes
1answer
15 views

How total loss is manipulated in mini batch gradient descent as the loss function is calculated and minimized for mini batches?

In mini batch GD, the loss function is calculated for mini batches. Suppose we have 480 training example and the batch size is 32. So there will be 480/32= 15 loss function. In every batches these 15 ...
1
vote
0answers
27 views

VAEs: Using Neural Networks To Approximate Conditional Distributions

Given the setup of a VAE, such as that outlined in Kingma and Welling (2014), there is a conditional probability distribution $p(z|x)$ describing the distribution of generated data $z \in \mathcal{Z}$ ...
9
votes
1answer
2k views

Which deep learning model can classify categories which are not mutually exclusive

Examples : I have a sentence in job description : "Java senior engineer in UK ". I want to use a deep learning model to predict it as 2 categories : English and <...
0
votes
1answer
12 views

Could ResNet curve be used for a regression problem, e.g. housing price prediction?

A Residual Neural Network (ResNet) is an Artificial Neural Network (ANN) of a kind that builds on constructs known from pyramidal cells in the cerebral cortex. Residual Neural Networks do this by ...
1
vote
1answer
229 views

Training a neural network with a training set with no noise

I am using artificial neural networks for an unconventional problem and, although it looks like it is working, I want to make sure that I am not doing anything wrong. I have a code, STARCODE, that ...
0
votes
1answer
25 views

Does a CNN have to be Fully Connected?

So I am trying to implement a specific CNN called a U-net. It states in page 3 that it doesn't have a fully connected layer. Till then I understood CNN to have two stages; 1. Convolution, where the ...
0
votes
0answers
13 views

Deep learning ; LSTM out-of-sample prediction

I am trying to do out-of-sample prediction of housing price index with deep learning LSTM. I've practiced the code with a sample data (apt_data_sc) splitting it with 70%,30% training and test set (...
0
votes
0answers
5 views

Best model (epoch) based on valid loss or valid evaluation metrics

This is a general question for how to select the best model after we finish the training process. In the training process, I always draw two plots: one is train/valid loss ~ epoch, the other is train/...
0
votes
2answers
48 views

In a neural network, why can't there be more weights than the number of observations?

After having this exact same issue with caret, I arrived at this thread. However, I do not intuitively understand why this answer is correct. Why can't there be ...
1
vote
1answer
405 views

Q-Learning using ANN with continous action and variable-length state

The question is basically What should I do if state vector has variable length? If the action is bounded and continuous, how can I obtain max(Q(state,action)) without using painfully slow global ...
0
votes
1answer
33 views

Glorot/ Xavier Init: for sigmoid and tanh?

My question is about Xavier Glorot Init. The assumptions that they make are that they approximate the activation function linearly, that this function has f'(0) = 1 and that we set the bias to 0, as ...
0
votes
0answers
9 views

Feature contribution for black-box methodologies

I am currently working on a research case involving a time series data where I want to make use of feature contribution (feature contribution can be explained as parameters for the features/variables ...
2
votes
1answer
26 views

Training loss goes up and down regularly. Weight changes but performance remains the same. What should I do?

As you can see in the picture, when the loss reaches 1.54, it goes up and then drops to the same number again and again. But if I reduce the learning rate, the loss is maintained around 1.6, and goes ...
0
votes
0answers
6 views

Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task: BERT might perform ‘feature extraction’ and its output is input further to another (classification) model The ...
43
votes
5answers
35k views

Difference between “kernel” and “filter” in CNN

What is the difference between the terms "kernel" and "filter" in the context of convolutional neural networks?
0
votes
1answer
177 views

ranking neural net models with feature selection

I have a sample with around 2000 observations and 10 variables which im using for classification. I plan on classifying the data with a neural net, but before doing so im using Weka's attribute ...
0
votes
0answers
16 views

How to bring the pre knowledge C into the training? [closed]

There are 3 columns used to train to model, A,B and C. A is the predicted value by a model, B is the target value in the dataset, C is the pre knowledge. C is the evaluated value to B, the absolute ...
0
votes
0answers
9 views

Organzing data in csv table: ANOVA on brain network

I'm looking at functional connectivity in a brain network, called the default mode network, which has 10 regions of interest. Meaning, if I was to do a comparison between each of the 10 brain regions,...
2
votes
1answer
661 views

sentence tokenization with deep learning

Anyone have any key words I can google, or links, for tutorials regarding how to do sentence tokenization using deep learning? The machine learning MLE method looks like so much work you may as well ...
0
votes
0answers
7 views

Generating Fixed Length Sequence with RNNs

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
0
votes
1answer
35 views

How can I speed up the training time of my neural network

I am a beginner to ML and AI, so I apologize if this is a bad question or anything but I can't seem to find any solutions other than buying better components which I don't have the luxury to do. So ...
0
votes
0answers
3 views

What are good basic loss functions for audio generation? (TTS)

I'm planning to make an audio generation NN. While I'm reasonably ok with neural networks in general, wavenets, etc., something is not quite clear. What are good loss functions for audio, considering ...
0
votes
0answers
10 views

Should Training,test and validation data sets should be independent of each other in neural networks?

I have a dataset of n items and i draw three samples of equal size. These three samples are training set,test set and validation set. I want to use this data for training ,testing and to check ...

1
2 3 4 5
150