Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network ...

0
votes
0answers
27 views

How to include new data into existing algorithm?

I have a complex ensembel algorithm X (divide data with k means that learn ensembel for each subgroup). Learning time of X is approx. 20 hours. I cannot afford to relearn algorithm for every new ...
1
vote
0answers
19 views

Neural network regression: seemingly bounded output

I have been working on a neural network based predictor for a project. The aim is to learn a certain quantity, say the signal strength of a cellular network, for each coordinate set in the dataset. ...
0
votes
0answers
10 views

Neural network how to deal with comparison

I'm currently working on a DQN network and this question comes to me. As far as I know, neural networks are good at dealing with values that have never seen (generalisation). E.g. If a classification ...
1
vote
1answer
46 views

Can neural networks learn $g(x)$ from $\mathbb{E}[g(X_t)] = \int_{-\infty}^{\infty} g(x)p_t(x)dx$

Let $\mathbb{E}_x[g(X_t)]$ be the expected value of a random variable $X_t$ with known probability density $f_t(x)$ then for the continuous case $$\mathbb{E}[g(X_t)] = \int_{-\infty}^{\infty} g(x)...
1
vote
0answers
12 views

Faster RCNN - Pyramid of Filters vs Pyramid of Anchors (Reference Boxes)

I'm reading faster RCNN paper now and trying to understand what is the difference between Pyramid of Filters and Pyramid of Anchors methods from the scale point of view. I mean if I use only one ...
0
votes
1answer
20 views

Gradient in batch-size

When we set a batch-size, after each sample of batch passed we take the gradient but wait until last sample of batch to passed and then propagate the sum of gradient of them through the network? Am I ...
-1
votes
0answers
64 views
+50

How to interpret that my model gives no negative class prediction on test set?

I am making multivariate time series classification with TUH seizure corpus dataset I have built this model with Keras, using LSTM layers : ...
0
votes
1answer
23 views

Accuracy percent seems too high in Neural Network

I built an artificial neural network that has a dependent variable called "Suspicious". This column is binary so only two outcomes. I have 297,771 "0" not suspicious or known good. Then I have only ...
0
votes
1answer
30 views
1
vote
0answers
30 views

How to input a continuous distribution to a neural network

I have simulated the relative frequency of a stochastic process by creating a very small grid say $1000$ by $1000$. The graph looks like this Now I am trying to setup a regression model by ...
0
votes
1answer
31 views

How can a network with only ReLU nodes output negative values?

I'm trying to use an api with a feedforward neural network for time series forecasting. For dense aggregate data it works fine, but for sparse data it sometimes forecasts negative values, even though ...
0
votes
0answers
25 views

How to fight imbalanced data in regression task? [on hold]

Suppose a reggression task, where solution space is [0..1]. But our dataset has more examples of solutions closer to zero, than to one. I am training a neural network. It is biased to predict numbers ...
0
votes
0answers
22 views

Autoencoder predicts the same image always [on hold]

I have an autoencoder that looks like this: ...
0
votes
3answers
46 views

Object localization with CNN

I am interested in locating the center of a playing card on the surface of a table: I have written a script so that I can generate images like this, where the card is moved around and rotated. My ...
2
votes
1answer
30 views

How does neural network training work, if there are A HUGE number of points that not differentiable?

When I first saw ReLu function, I would not guess it will work in neural network because there is a point that is not differentiable. But it seems works very well on modern neural network. My ...
1
vote
2answers
15 views

Is it better to avoid ReLu as activation function if input data has plenty of negative values?

ReLu is probably the most popular activation function in machine learning today. Yet, ReLu function outputs 0 when input data values are negative. ReLu totally disregards negative data. This may ...
-1
votes
0answers
10 views

recurrent neural network [on hold]

I study on navigation systems and is interested in using neural network for GPS/INS during GPS outages.I implemented the integration of INS/GPS information using the KّFand the EKF. But I can not ...
0
votes
0answers
21 views

How to decide which activation function to use? ReLu or sigmoid? [duplicate]

Sigmoid function gives equal weightage to both negative and positive input values. ReLu function outputs 0 for negative input values and output follows the input for positive input values. Sigmoid ...
1
vote
0answers
16 views

GAN for learning the transition density of a Markov process

I have learned about the Generative Adverserial Networks and the way they are used for learning the underlying (complex) distributions of high dimensional data. Now, my question is: Are there ...
10
votes
3answers
206 views
+150

What *is* an Artificial Neural Network?

As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
0
votes
1answer
16 views

is there any diffrence to use nonlinear or liner activation function in single hidden layer network

I am working in a classification problem in which I use RBF with a single hidden layer. I want to use SoftMax activation function for the hidden layer. I already read some documents about the ...
-1
votes
0answers
8 views

Is it sufficient to normalize the input Data to perform a multiple Output Regression where the labels have different magnitudes?

I am trying to simplify a complex mathematical model in a certain range by performing a regression with a neural Network. I am using a hidden layer with a 'tanh' activation function to normalize my ...
1
vote
1answer
16 views

Adding the input layer - units with a decimal

I took the course Machine Learning A-Z from Udemy and am trying to apply what I learned in the tutorials. Theye taught us in the "Adding the input layer" portion of an ANN that the units is based off ...
0
votes
0answers
9 views

How are features learned during hidden layer better than quadratic features?

In Andrew Ng's course Machine Learning, Week 3, Model Representation 2, he mentions that features learned with a simple hidden layer can be better than simple linear relationship features and even ...
2
votes
1answer
35 views

Forecasting non-stationary time series using MLP

I noticed that in many tutorials with neural networks people difference their time series prior to training/forecasting. Suppose that we have a window model with many autoregressive terms (say 365 ...
0
votes
1answer
19 views

Cross Entropy calculation question: calculated is different from Keras' output

I wrote a simple code to test Keras cross entropy, but got different results from this post. I checked everything, but still do not know why keras gives me ...
1
vote
1answer
25 views

Does it make sense to use a dropout layer in a neural network for a regression to predict an absolute Error?

I am working on a regression problem where I try to predict an Error with a NN with as little calculation steps as possible. Currently I have an input layer consisting of 21 Neurons and a Dense Output ...
1
vote
1answer
76 views

Time-series prediction with RNNs: What to expect from the learning process?

When training an RNN for time series prediction, what can one expect to see visually as the model learns? In particular, are plateaus a normal indication that the model is underfitting or do they ...
1
vote
0answers
32 views

Machine Learning on Extremely Low Signal Data

I have terabytes of data with an extremely low signal to noise ratio, with the following characteristics: The relationship between the features and the response variable can change over time I'm ...
0
votes
0answers
28 views

ANN: Total weight changes in the back-propagation rule

Preliminaries I am writing an MLP. In this book one can find description of the back-propagation learning method. Starting with the feed-forward ANN one changes weights during learning according to ...
-1
votes
0answers
12 views

Recommendation System using Recurrent Neural Networks

I am trying to build a video game recommendation system based on amazon video game user reviews. I am using the following paper as a guide: https://cs224d.stanford.edu/reports/LiuSingh.pdf. So far I ...
1
vote
0answers
33 views

Keras NN - loss gets stuck at 8.6791

What does it mean when my neural network always gets stuck at the exact number 8.6791 when I use binary-crossentropy loss? Some strange local minimum? It happens regardless of my learning rate, ...
1
vote
0answers
22 views

Adverserial Verification of an XGboost Classifier

This paper proposes an algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The ...
0
votes
0answers
26 views

ResNet18 Implementation

Disclaimer: Though my question uses code, my question is not specifically about how to get the code to work as I am not actually trying to train a model. So as a working exercise, I'm looking to ...
2
votes
1answer
24 views

Does it make sense to use an Early Stopping Metric like “mae” instaed of “val_loss” for regression problems?

I am performing a regression on a Dataset and try to replace a mathematical Model with a Neural Network. To avoid overfitting I decided to use the Early Stopping Callback Function of Keras. So far I ...
2
votes
1answer
24 views

Extensions of LSTM for huge data

Consider dealing with a huge high frequency financial data forecasting, RNN/LSTM is a popular way to solve the task. But the problem is that say you have 1 million data points and you want to predict ...
3
votes
1answer
37 views

What is an “undirected associative memory” in Hinton et al 2006?

In A fast learning algorithm for deep belief nets, the authors use the term "undirected associative memory". I am not sure what they are referring to, and unfortunately Google searches for this term ...
2
votes
0answers
27 views

NeuralNetwork does not converge given a DataSet [duplicate]

I have problems teaching a Neural Network to learn a game. The following NeuralNetwork does not converge suitably. In the "Game" the player has to move a red dot by applying a force to it. a human ...
0
votes
0answers
15 views

How to use machine learning to detect users based previously written texts? [closed]

I am new to machine learning and I have a task to create a system, which will be able to detect/recognize users based on what they previously wrote. After training, the system will receive new ...
1
vote
1answer
23 views

Proof for the efficiency of Softmax in multi-classification

I already search for this question but I can't find any convincing explanation so I want to ask it here. my problem is with softmax activation function and cross-entropy.why they can produce a better ...
4
votes
0answers
26 views

Weight normalization technique used in Image Style Transfer

I am trying to implement the paper Image Style Transfer Using Convolutional Neural Networks. In section 2 - Deep image representations, the authors mention the following weight normalization technique:...
2
votes
0answers
14 views

Lack of Batch Normalization Before Last Fully Connected Layer

In most neural networks that I've seen, especially CNNs, a commonality has been the lack of batch normalization just before the last fully connected layer. So usually there's a final pooling layer, ...
0
votes
0answers
19 views

R neuralnet package is very slow for classification project [closed]

I am running neuralnet function to create classification model on a data set, which has 21 input variables, one response variable with two possible classes and around 3400 values. I transformed it to ...
1
vote
1answer
48 views

Sampling z in VAE

How many times do we sample from $Q(z|x)$ in a Variational Autoencoder? Let’s say that the autoencoder input $x$ is a single image 28x28 pixels - and $Z$ is is a one dimensional distribution. Then, ...
2
votes
1answer
41 views

How to handle big vocabulary size with keras tokenizer?

I am actually working on a neural language model developed with keras. I have an encoder and a decoder and the output of the decoder is a dense vector on the vocabulary..so quite big depending on the ...
0
votes
0answers
14 views

Avoiding OCR performance coupling to upstream Bounding Box model

I have a model pipeline where I first use an object detection deep learning model to locate text regions in images of natural scenery (i.e. outdoor images), and then send the cropped region to a deep ...
0
votes
1answer
17 views

How to correctly retrain model using all data, after cross-validation with early stopping

I have a classification task that doesn't have loads and loads of data, so I'd like to make the most of the data. I have a boosting model and I've performed 5-fold CV, using the validation fold for ...
1
vote
0answers
15 views

Adaptive Moment Estimate - What is meant by parameters of Adam optimizer are biased towards zero initially?

Let us consider the Adam optimizer with the equation given below: $w_{t + 1} = w_{t} - \frac{\eta \times \bar{m_t}} {\sqrt{\bar{v_t} + \epsilon}}$ Here $w$ denotes weight (in time $t$ and $t + 1$) ...
0
votes
1answer
27 views

Are folds changed between epochs in K-Fold Cross Validation?

2 Related question about Cross-Validation (In the scope of Neural Networks): 1) Let's say we train our neural network for 100 epochs and apply 5-fold cross validation. In that case, should I use the ...
1
vote
1answer
62 views

Poor regression results of neural networks on 2d benchmark data (compared to spline interpolation)

I try to understand for which regression tasks neural networks might be useful. One benchmark for me is to reproduce the ability of scipy.interpolate.griddata: ...