Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
1 vote
0 answers
17 views

Neural Network doesn't work with my shape

If I have for example dataset with input shape (768,768) and output (4). And if I give for input shape(number_of_examples, 768, 768) and for output shape (number_of_examples, 4), it will not work. ...
user avatar
0 votes
0 answers
5 views

Shuffle data inside learning sample in order independet transformer model

Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
user avatar
1 vote
1 answer
18 views

Not improvement in representation of encoder in AE

I am trying to train an autoencoder on tabular data containing categorical data. After training AE, I use the encoder for classification. I normalize numerical data and use one-hot encoding for ...
user avatar
8 votes
6 answers
1k views

Success Stories of "Statistics"?

When it comes to Machine Learning, the successful application seem to be very well known. For example, Neural Networks have been successfully used to create Self Driving Cars and Game Playing ...
1 vote
0 answers
15 views

Can I use batchnorm in CNN + RNN, and where to place it exactly?

I have designed a following neural network that combines CNN, RNN and Dense layers. It aims to predict a positive or negative outcome for the time step t+1, given a ...
user avatar
  • 181
0 votes
0 answers
9 views

How to detect multi-user time series outliers

My data is three-dimensional data with id, value, and time. ID DATE_TIME VALUE ID 1 2020.01.01 10 ID 1 2020.01.02 8 ID 1 2020.01.05 15 ID 1 2020.01.06 12 ID 2 2020.01.02 20 ID 2 2020.01.17 22 ...
user avatar
  • 3
1 vote
0 answers
7 views

CycleGAN cycle loss

I was reading the paper of CycleGAN and I was trying to implement it. However, my models does not converge to any good solution whatsoever, and since I've checked the implementation many times, I ...
user avatar
0 votes
0 answers
22 views

Categorical loss function for variable number of labels [closed]

I have a model for binary classification. The target variable has the different number of labels (instances) in each sample. For example, a batch of size 2 with 2 and 3 instances and correspondingly ...
user avatar
0 votes
0 answers
23 views

Survival analysis for interval censored data using deep learning

Apologies if I mess something up, I've only used Cox PH once and am only starting with deep learning! In my project, I'm trying to estimate the best interval for screening patients for some ...
user avatar
0 votes
0 answers
29 views

Improve the accuracy of one class, at the cost of other classes

Say we have a balanced dataset of two classes of objects: green light, red light. After running deep NN classification, the model gives about 95% accuracy for both classes. However, I need to increase ...
user avatar
  • 719
1 vote
1 answer
23 views

How should I train my CNN with a tiny dataset

I'm working on a problem where I aim to classify sections of a track made on the floor using tape, into either left turns, right turns or straight track. I'm struggling creating a CNN that is not ...
user avatar
0 votes
0 answers
7 views

what does it mean if all my deep learning models have the same precision

I have 4 different deep learning models that have different accuracy different recall and f1 measure but have the same precision what could be the reason for that?
user avatar
0 votes
0 answers
10 views

What's GAN's Input-size Limitations?

I am interested in GAN for generating synthetic data. I am studying the input limitations for GAN starting from which GAN is no longer usable. I have found many applications that use GANs for ...
user avatar
  • 101
0 votes
0 answers
5 views

Conditional GAN training

Quite a simple question about this paper: How is the cGAN trained? I'm interested in the Pix2Pix network In particular, given the batch approach, given a specific step, is this the correct procedure? ...
user avatar
0 votes
0 answers
7 views

does kernel or filter update through backpropagation , is require fully connected layers?

does CNN itself without FC has the ability to make backpropagation to update its filters, after comparing them with the output and calculating the loss, and then pass them through the FCC?
user avatar
-2 votes
0 answers
12 views

Different modes of reasoning in AI [closed]

Is data science, AI( machine learning, deep learning) deductive or inductive reasoning
user avatar
  • 101
1 vote
0 answers
11 views

Equivalency of feedforward and recurrent neural network

I'm involved in solving a problem as described below and wondering if you could kindly help me figure it out: "approximating a single input single output single layer with two neurons , ...
user avatar
  • 11
0 votes
0 answers
14 views

Pix2Pix facede dataset, prevent "gray" in dataset to be predicted

I'm trying to build from scratch the pix2pix architecture, the one on this paper. As they did, I'm using the facade dataset, and this is one of their result: I'm particularly interested in the last ...
user avatar
1 vote
2 answers
11 views

What is the difference in ANN with lagged observation as predictors and LSTM?

I am trying to understand how ANN is different from LSTM when we include lagged variables as predictors. Is the difference solely in forget gate for LSTM?
user avatar
  • 11
0 votes
1 answer
9 views

GAN artifacts on borders

not quite a math-question, but I have a doubt. I'm trying to build from scratch the Pix2pix network, on the facades dataset, and I think I finally got a good model (from the paper I borrowed just the ...
user avatar
0 votes
0 answers
9 views

Cumulative accuracy of two models in sequence (Neural Networks)

I'm looking for an accuracy formula given two different networks which work together in a system. The problem is the following: There are two networks, one which divides the dataset into two subset ...
user avatar
1 vote
2 answers
32 views

How does epoch-wise double descent occur if training error is 0?

In this paper, they talk about the existence of epoch-wise double descent. In Figure 10, you can see that, with a sufficiently large model, the test error keeps decreasing even after the training ...
user avatar
1 vote
1 answer
9 views

Lowering the weight of particular features in a neural network?

Given sample data $x$, we hypothesize that some features (i.e. dimensions) of $x$ will generalize well, while others will generalize poorly. For example, when predicting medical diagnosis, age and ...
user avatar
1 vote
0 answers
19 views

Can a neural network fail to replicate the training data if trained on a very small dataset?

I created a neural network and I have trouble getting it to train. I followed all advice in this post: What should I do when my neural network doesn't learn? but have not had success yet. Then I ...
user avatar
0 votes
0 answers
15 views

Cross Entropy with a Very Sparse Output

I'm interested in training a deep convolutional network for 2D gridded data that, instead of classifying an entire sample, will classify each pixel in the sample. I'd like it to find the center pixel ...
user avatar
  • 111
1 vote
0 answers
31 views

What is the correct way of normalizing/standardizing image-like data?

I have image-like data (e.g. H x W x C), where each channel contains quite different information. You can think of it being a 2D map (H x W) with information like elevation, wind velocity, temperature ...
user avatar
0 votes
0 answers
13 views

Can increasing dimentionality improve classification in Neural Networks?

I had a dataset where each data sample (pixel) had 7 features (reflectance at 7 wavelengths). However, running my neural network on the 7 features was not able to reach a high accuracy in classifying ...
user avatar
  • 111
1 vote
0 answers
66 views

Predict parallel time intervals

I have the following problem. There is a service station that can provide service for a number of vehicles at the same time. The service data looks like this: ...
user avatar
2 votes
1 answer
38 views

Best hyperparameter is not consistent among different seeds

I do hyper-parameter tuning on my network and it outperforms the simple classifier. The difference in classification is considerable after hyper-parameter tuning. But, the problem is that an optimal ...
user avatar
1 vote
0 answers
22 views

Check if the embeddings preserve the true space structure

I obtained the embeddings from the pre-classification layer of a neural network for 600K instances. Each embedding's dimension is 1024, thus we have a sample of the embedding space, $\mathcal{X} \...
user avatar
  • 241
0 votes
0 answers
12 views

Thermodynamic modelling of Salt curve using neural networks

I am trying to achieve a predictive thermodynamic model of the Salt curve for different surfactant solutions. Data For the same is available for different surfactant/salt/additive mixtures of various ...
user avatar
1 vote
0 answers
58 views

What is the maximum Q-value that any state can obtain in DQN?

The q value of a specific state,$s$ and action, $a$ is given by the following equation, as per Sutton and Barto's equation 3.13 - $$q_{\pi}(s,a) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty}\gamma^{k}R_{t+k+...
user avatar
1 vote
0 answers
11 views

consistent gap between training and validation metrics

I am training a neural network (Deep and cross network) for a multi-label classification task (~700 labels). I am seeing a uniform gap between training and validation results on various metrics. E.g. ...
user avatar
  • 311
6 votes
1 answer
46 views

Standardizing neural network inputs with a linear layer?

I'm contributing to a ML software project and noticed something weird in the code: They perform standardization by introducing a linear layer right after the inputs. This linear layer has the same ...
user avatar
1 vote
0 answers
19 views

Why does a neural network have the same output for every item in a batch?

I am trying to train a small MLP in Pytorch. Here is the code for the net: ...
user avatar
  • 11
1 vote
1 answer
30 views

When the SVM will be a better option to using than CNN

I want to use a dataset with five features related to three classes ( normal, UDP, and SYN) to detect the DDoS attack, In this idea, which algorithm should be used ( ML with SVM or DL with CNN) and ...
user avatar
  • 11
0 votes
1 answer
30 views

The effect of over-parameterization on local minima

While reading some papers about over-parameterization in deep learning models, I also read that "over-parametrization is a simple method to introduce additional dimensionality and help make the ...
user avatar
  • 1
2 votes
1 answer
18 views

Can we combine different yolov5 models trained of different classes into one?

Suppose I've a yolov5 model trained on cars and second trained on bus and third trained on bike and so on. Is there a way through which I can combine all the model into a single model? As by running ...
user avatar
0 votes
0 answers
8 views

plotting the latent space of a GAN

I am working on gans and wanted to know how I can plot the latent space of gan. Like I have a latent space of shape (50,250). So it is an n-d array of length 50 and 250 points representing each one of ...
user avatar
  • 1
0 votes
0 answers
31 views

Calculating the variance of softmax

I'm working through Dive Into Deep Learning right now and am struggling with the following question: We can explore the connection between exponential families and the softmax in some more depth. ...
user avatar
1 vote
0 answers
9 views

Why not using Convolutional pooling instead of MaxPooling to avoid invariance

I was reading the paper of [Geoffrey Hinton: Capsule network], and I watch it's talk on Youtube about the problem of Conv Network is actually the (max) pooling layer, since we don't want to be ...
user avatar
0 votes
0 answers
35 views

Gradient descent / Adam converging to suboptimal solutions

I am using neural nets to find the minimum of a complex function to which I compute the mean (crit in my code). Here is my net : ...
user avatar
  • 1
3 votes
1 answer
87 views

Variational Autoencoder not able to reconstruct outputs, though an Autoencoder with a similar architecture works

I am trying to use a variational autoencoder-like architecture that converts images of a dataset that I created myself to an equivalent compact representation. Below is my code for the model ...
user avatar
  • 89
0 votes
0 answers
10 views

Preparing dataset over clients in federated learning

I am working on google cluster trace. I am working on small resource usage data only for 100 physical machine. Sample of data for two machine: To predict the resource usage I used classical ANN. Now ...
user avatar
0 votes
1 answer
26 views

Is Hessian of neural nets with NLL loss positive semi-definite?

I learned that expected Hessian of negative log likelihood is the same as Fisher information matrix, which is known to be positive semi-definite $$ \begin{aligned} F(\theta) &= E_{x \sim p_\theta}...
user avatar
0 votes
0 answers
8 views

Can I deal with datasets of images with different sizes with pre-trained densenet?

I have a dataset of images with very different sizes, but that are bigger than 224 X 224. It seems that the pre-trained densenet model of pytorch can accept images of different sizes during training ...
user avatar
  • 504
0 votes
0 answers
20 views

Transformers for correcting single word misspellings

I'm asking for your kind help to know if there is some known strategy/reference to use a transformer-like model to solve the following problem: The input is a single misspelled word, such as: $$dta$$ ...
user avatar
7 votes
1 answer
298 views

What happens when DQN gradients become too big?

I am reading these notes on slide 34 and came across strategies to prevent gradients from becoming too big in Deep Q Learning (DQN). Since, we don't usually use deep architectures in DQN, I don't ...
user avatar
1 vote
0 answers
34 views

How ResNet researchers achieve high accuracy?

I tried to recreate the original ResNet 50 model myself, using all the same layers, hyperparameters, and data augmentation methods mentioned in the paper. (I only trained on a few hundred iterations ...
user avatar
-1 votes
0 answers
34 views

Neural net backpropagation formulas with multiple hidden layers

I am trying to figure out how backprop works when dealing with multiple « ways » to calculate the gradient. (Linked to https://ai.stackexchange.com/questions/31566/different-ways-to-calculate-...
user avatar

1
2 3 4 5
189