Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
2
votes
0answers
10 views

What are the non-differentiable neural network architectures?

Neural networks are generally not differentiable (in the rigorous mathematical sense) due to activation function such as ReLu Recently I have been wondering about the other possible sources of non-...
0
votes
0answers
32 views

Do regression algorithms generalize better than classification Algorithms? [closed]

I apologize if this question sounds a bit vague, but I would appreciate any thoughts/insights that anyone may have: Suppose I have two Deep networks; one is trained for multiclass classification, and ...
-1
votes
0answers
11 views

Replacing Random Forests with CNN [closed]

Supposing I have a set of optical images to classify in for ground occupation issues. this classification is done using some Random forest algorithm with iteration. I mean by that that each time the ...
5
votes
0answers
28 views

Local optima in high-dimensional optimization

I remember a theorem a long the lines of In higher dimensional optimization problems, you are less likely to get stuck in local optima, because the more dimensions you have, the more likely you are ...
2
votes
1answer
21 views

Overview Feature Extraction in images?

I have been searching for deep feature extraction approaches for a while now, but I did not find a single paper giving me a coarse overview on this matter. Apart from an overview, for example I would ...
1
vote
1answer
29 views

merge two neural networks at test time

I am currently using one network which gives me a bad resolution result, so then I use another network to enhance the resolution of my output. My question is: is there an easy way to use both of them ...
0
votes
0answers
9 views

Which model to use? (cross validation with early stopping)

In this example, to keep things simple we use only 1 training and validation set, and we are trying to find the best regularization parameter for ridge regression. The square loss below is on the ...
0
votes
0answers
26 views

DNN underestimates high values

I'm running a DNN on a dataset in order to predict the output values (y). The actual vs fitted graph shows a slight overestimation of the small values and an underestimation of the higher values. The ...
0
votes
0answers
32 views

Backpropogation Implementation [closed]

I have been working to understand the backpropogation algorithm, and I finally think I have gained some grasp of the concept. So I have now begun trying to implement the algorithm, and have so far had ...
0
votes
0answers
12 views

What activation and where to use in MaskRCNN RPN

so I've been trying to implement my own version of MaskRCNN, and I am baffled by how the RPN is implemented in various places. Assuming the standard RPN architecture of a shared 3x3 Conv2d, and two ...
0
votes
1answer
23 views

How to make use of ground truth data in image anomaly detection?

If I have an image dataset that consists of "normal", anomalous and ground truth image data, how do I make use of the ground truth data? To my understanding if I train an unsupervised ...
1
vote
1answer
14 views

Train loss value to consider convergence

I am training a fully connected NN with 4 hidden layers for a task of regression of two rational target values, Using MSE loss. My problem is determining whether the training process succeed, that is, ...
0
votes
0answers
25 views

How do they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
0
votes
0answers
6 views

Normalization and Standardization of color channels for Convolutional Neural Networks

I have created 2D heat maps with 3 color channels. On these heat maps, I will train CNN networks. The range of values in the three colors channels is very different. In the first channel the values ...
2
votes
0answers
15 views

Confusion about the derivative in CTC

I was going through the original CTC paper by Graves et al, I am still not getting how after taking the derivative of equation 14 we get equation 15 as shown below I understand the part that we are ...
0
votes
1answer
15 views

Hyperparameter tuning vs weight tweaking in Cross-Validation: should I consider 2 different validation sets?

Let's say I have 1000 Samples and want to build an ANN. Then I split my dataset into train set (800) and test set (200). After that, I do the following Cross-validate my train set with different ...
1
vote
0answers
25 views

Why do CNNs work well with natural data such as speech, images, and text?

According to the universal Approximation theorem, we can approximate any given function with two-layer neural networks with a sufficient number of nodes. Then Why do CNNs work well with natural data ...
0
votes
0answers
11 views

Understanding the decision boundary using a tanh activation function in a Neural Network

I was hoping that someone might be able to explain to me a bit why the decision boundary looks the way it does. I believe that this has to do with SGD and this reflects the derivative of the tanh. ...
1
vote
1answer
9 views

ReLU outperforming Softplus

I have noticed that PyTorch models perform significantly better when ReLU is used instead of Softplus with Adam as optimiser. How can it happen to be that a non-differentiable function is easier to ...
1
vote
0answers
6 views

How to define loss function for Discriminator in GANs?

To train the discriminator network in GANs we set the label for the true samples as $1$ and $0$ for fake ones. Then we use binary cross-entropy loss for training. Since we set the label $1$ for true ...
0
votes
0answers
8 views

What should be the ratio of number of classes to number of instances per class?

I am trying to train a CNN model for the classification of 100 different classes. I have about 275 instances for each class and there are about 1000 features. While I trained the model by tuning ...
0
votes
1answer
8 views

Cross-validation for hyperparameter tuning

I've read as many topics regarding hyperparameter tuning as I could, and I developed the following algorithm for hyperparameter tuning & final model building Split the data in train set (80%) &...
0
votes
0answers
10 views

what is Linear Bottleneck? [closed]

How does Linear Bottlenecks work in MobileNetV2?
1
vote
1answer
30 views

Aging/oldify a dataset of pictures

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) My team and I are interested in reproducing their work. However, ...
0
votes
0answers
8 views

Image intensity normalization in preprocessing

Suppose having two images on a given scale, for example it could be the classic [0-255], representing the same thing but with different value intensities, i.e. the first could have a maximum pixel ...
0
votes
0answers
7 views

ML model type for making multi-step time-series predictions

Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices. What would be the best choices in terms of model structure for this problem? I have considered ...
0
votes
0answers
11 views

Faster RCNN for one class in object detection

Let say I have a task to detect the bounding box of one object only. And the only thing I care about is the IoU between prediction and ground truth, no need for real-time. My question: Should I ...
0
votes
0answers
4 views

ML training and test data in making multi-step time-series predictions

Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices. What would be the test data in this problem? The 1 month I would make a prediction for, or ...
0
votes
0answers
11 views

why Alexnet fails even on slight modification to layers [duplicate]

This lines are found in discussion section of Alexnet's Paper. Our results show that a large, deep convolutional neural network is capable of achieving recordbreaking results on a highly challenging ...
6
votes
2answers
172 views

Best way to reduce false positive of binary classification to exactly 0?

I'm working on a task that even a 0.00001 fp rate is not acceptable, because detecting something as a positive when its not will have very bad consequences in this task, so it needs to be exactly 0 ...
0
votes
0answers
13 views

Output and Input Size of CNN

I have a convolutional layer $g$ with 10 feature maps given by: $$g(x^i) = \sigma([z_1,z_2,\dots,z_{10}])$$ where $z_j = f(x^i,w_j)$ is the output of a 1-d convolutional operation parameterized by a ...
0
votes
0answers
8 views

Are there defined categories of features in images? [closed]

I have been looking into deep anomaly detection and I am currently wondering about what kind of features can be extracted from an image. I have seen papers about edge features and texture features, ...
0
votes
1answer
17 views

FInding number of trainable parameters in CNN

I have a convolutional layer $g$ with 10 feature maps given by: $$g(x^i) = \sigma([z_1,z_2,\dots,z_{10}])$$ where $z_j = x^i \cdot w_j$ for some convolutional kernel $w_j$ of size 3. Each $x^i$ is ...
1
vote
1answer
33 views

What to do when a neural network cannot overfit one training sample? [closed]

Other questions have addressed what to do when a network does not reach good performance on a (medium / big) training set or that overfitting one training sample requires enough capacity. However, ...
0
votes
0answers
12 views

Hyperparameter Tuning Keras Functional API [closed]

Is it possible to conduct hyperparameter tuning on a keras functional API? Such as using gridsearch or randomsearch? I am aware it's possible with Sequential function but can it be done directly with ...
0
votes
0answers
22 views

Why scaling down the parameter many times during training will help the learning speed be the same for all weights in Progressive GAN?

The title is one of the special things in Progressive GAN, a paper of the NVIDIA team. By using this method, they introduced that Our approach ensures that the dynamic range, and thus the learning ...
0
votes
0answers
15 views

When to stop training a binary classifier when precision-recall curve and logits output is of interest

I have a binary classifier (DL model) and a balanced dataset (50-50 for the two classes, in both training and testing data). Let's say I want to optimize for precision. I can adjust my positive label ...
1
vote
1answer
10 views

How to build a tokenizer using Neural Networks?

I’ve been trying to build a NN tokenizer where the inputs would be chars and the outputs, tokens. But it is not clear to me how this kind of model should work in terms of the output format. If the ...
0
votes
0answers
9 views

latency not decreased tf-lite post training quantization

I am using efficient-net to classify images. I have trained model successfully and wanted to quantize it using tf-lite. I tried all the methods available in tf-lite quantization to check accuracy, ...
0
votes
1answer
19 views

Transfer Learning on Autoencoders?

I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework. For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-...
1
vote
0answers
5 views

How Restricted Boltzman Machine (RBM) generates hand-written digit?

I am reading RBMs from this paper. In Fig1 they show an example of generating hand-written digit using RBMs. This is the figure they are showing: In the learning step first we sample $h$ from $h \sim ...
3
votes
0answers
24 views

Adding Laplace noise to a learned neural network

My question is related to the concept of differential privacy and deep learning. I found many papers to learn neural networks with differential privacy, but is it also possible to achieve differential ...
6
votes
0answers
38 views

Overfitting and underfitting in Neural Networks: Is total number of neurons or number of neurons per layer more relevant?

I have seen posts where the discussion was centered around the effect of big and small total number of neurons in a neural network, especially with respect to the potential of the network to overfit ...
2
votes
1answer
27 views

Overfitting small dataset necessary for deep NNs when training with big dataset works?

In the CS231n course from Standford, they state that a network should be able to overfit a small dataset by getting zero cost, otherwise it is not worth training. However, what if a network is not ...
1
vote
1answer
25 views

Are Deep Anomaly Detection Approaches able to tell me what kind of anomaly it is?

I recently started to look into anomaly detection. The deep learning approaches are trained on "normal" classes to build a classifier that can detect outliers (anomalies). Are these ...
2
votes
0answers
45 views

Can PCA generate a new random image?

I read The Batch: GANs newsletter and Goodfellow said: My colleague Bing Xu modeled face images from the Toronto Face Database, which were only 90 pixels square and grayscale. Because the faces were ...
0
votes
1answer
28 views

Deep Learning: Difference at the end of epoch iteration? [closed]

I read this thread. I have a follow up question. As I understand it there is no difference at the end of an epoch or an iteration . To be more clear, at the end of both iteration and epoch changes ...
0
votes
0answers
16 views

Assessing the performance of a neural network in classification task that involves more than two class types [duplicate]

In terms of evaluating how well a neural network performs in a classification task with the number of classes greater than 2 (for example, classifying an observation into one of the 4 classes), which ...
1
vote
1answer
16 views

Fully connected Recurrent Neural Network: question about full connectivity

Here is the picture of RNN It's said, that hidden layer is fully connected(dash dots). But I don't understand why? For example I don't understand why the one from $a_3$ to $a_1$ exists. I thought ...
1
vote
0answers
15 views

Effect of weight decay on loss [duplicate]

I am new to deep learning and the terminology "L2 regularization" and "weight decay" seems to be used almost interchangeably... L2 regularization however modifies the loss function ...

1
2 3 4 5
163