Skip to main content

Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

If we need the mean and variance of the entire batch for batch normalization, how do we do a single forward pass before we get these values?

If I understand correctly the model has various hidden layers with batch normalization layers in between. When trying to compute a forward pass to back-propagate, how do we know what mean and variance ...
user420414's user avatar
0 votes
0 answers
9 views

How can I use a neural network to choose a subset of features from a dataset? [duplicate]

Suppose my dataset has 256 features. Right now, what I can think of is this: Create an NN model like this: a. create a sequential model b. add a Conv1D layer c. add a flatten layer d. add 1024 dense ...
user366312's user avatar
  • 2,201
1 vote
0 answers
6 views

energy efficiency and time for a SIMD broadcast systolic data flow deep neural network

Im trying to understand why memfetch is multiplied by km(L+1)/8 instead of NMACS which is 8 and also what is meant by systolic clk increment Consider a fully connected layer. Let • X ∈ RK×L real 32bit ...
Mohamed Insaf's user avatar
0 votes
0 answers
13 views

B-Splines in KANs: How Are They Implemented?

Can someone help me understand how Kolmogorov-Arnold Networks (KANs) make use of B-splines? They talk about spline grids in the paper without explicitly defining what they mean by it AFAIK. For ...
Transcendental's user avatar
1 vote
1 answer
21 views

What would be the convolutional layer output by keras.layers.Conv2D when conv output is fractional?

I have input ($n=224$), strides ($s=4$), filter size ($k=11$) and no padding which gives me a fractional conv output: $$\texttt{conv output} = (n-k+2p)/s + 1 = 54....
Shri's user avatar
  • 13
1 vote
0 answers
25 views

Output from Model A as Training Data into Model B

Not sure this is the right place to ask this question, but I'm having a disagreement with a colleague on this idea. Let's say we have a dataset comprised of "unclean" strings. The end goal ...
setty's user avatar
  • 111
0 votes
0 answers
33 views

cannot identify important feature from global heatmap of shapley value

In my case, I have 166 features for each instance. I have split the train and test datasets in an 80:20 ratio and trained a DNN model for binary classification. The model architecture is as follows: ...
Stiven Choking's user avatar
0 votes
0 answers
16 views

How do B-splines differ from Fourier transforms?

I read that B-splines can be used as activation functions in KAN neural networks, whereas Fourier transforms are not widely used. Can someone please explain the difference between the two in a simple ...
Hughie Phan's user avatar
5 votes
1 answer
326 views

What is the best epoch to evaluate the test images?

I created a training, a validation and a test set for an image classification task. Then, I did training using the training and did evaluation on validation set. So, the next step is to evaluate the ...
cancan's user avatar
  • 53
1 vote
0 answers
35 views

How to Estimate GPU Memory for training and inference, Data Requirements, and Training Time for Large Language Models?

Today, I faced this question during an interview for an ML Engineer position. I didn't answer it perfectly at the time. How should I answer it ideally? Assume we have models like Transformer, BERT, ...
maplemaple's user avatar
0 votes
0 answers
39 views

Ways to parametrise a positive parameter

I am working with a differentiable state-space model involving a noise variance term $\sigma^2$ which I want to parametrise based on some features, e.g. $\sigma^2 = g(X\beta) > 0$, wherer $\beta$ ...
Danny Duberstein's user avatar
0 votes
0 answers
6 views

Efficient pooling to extract global embedding from local features (for LiDAR point clouds)

Problem: I have 3d point cloud data (autonomous driving setting) and a point cloud encoder (MinkUNet) that extracts local features from them. What are suitable pooling techniques to map those local (...
Hölderlin's user avatar
2 votes
2 answers
36 views

how long should i run a training to realize how well an NN model is doing?

Suppose I am manually tuning the hyperparameters of an NN model. How many epochs of training should I run at a minimum to realize that the model won't give me the desired accuracy I need before ...
user366312's user avatar
  • 2,201
1 vote
0 answers
21 views

Modeling for a data set that has different number of factors for each row (not binomial)

The modeling issue I'm having is that the categorical variable for each row has different number of factors. If I can reshape the data by products (a,b,c,.....~cost, hoursum, numPod, numDate), so that ...
rocknRrr's user avatar
  • 121
1 vote
1 answer
42 views

Weight initialisation for neural networks - should they be different for each observations or the same?

I am implementing myself a Neural Network with feedforward and backpropagation with gradient descent to understand better how things work. After setting up the entire algorithm, I still have a huge ...
umbe1987's user avatar
  • 297
2 votes
1 answer
27 views

The meaning of linear transformation in a batch norm revisited

I'm reading BatchNorm Wikipedia page, where they explain that BatchNorm. I think the actual formulas are easier than words in this case. The norm statistics are calculated as: $$\large{\displaystyle \...
Mah Neh's user avatar
  • 174
1 vote
1 answer
39 views

How can different models based on different sets of predictors be combined to significantly improve the model performance?

I have two machine learning models for predicting some continuous variable $y$, say $y=f_1(X_1, \theta_1)$ and $y=f_2(X_2, \theta_2)$, and these models are of the same type (ANN). $X_1$ and $X_2$ ...
tunar's user avatar
  • 523
1 vote
1 answer
14 views

NeRF vs mesh for text-to-3d generation

There seem to be multiple aproaches to generating 3d objects from text prompt. What's confusing is that some of them are generating NeRFs (https://arxiv.org/pdf/2308.16512), other's are generating ...
zlenyk's user avatar
  • 186
0 votes
0 answers
14 views

Suggestion for training various models for classification of different species

Working on developing classification networks for various types harmful algae vs algae/other objects found in the ocean water. We have developed binary networks for some harmful algae vs Ocean. There ...
Dhruvin Naik's user avatar
-1 votes
1 answer
23 views

How does a neural network differentiate between a neuron that outputs 0 and a dropped-out one?

How does a network differentiate between a neuron with output 0 and a dropped-out neuron (this neuron might output a non-zero value but due to dropout it outputs 0)?
ado sar's user avatar
  • 477
0 votes
0 answers
15 views

the Detailed Architecture of EfficientNetV2-B2

I'm currently studying different neural network architectures and I'm particularly interested in EfficientNetV2-B2. I understand that this model is an improved version of the original EfficientNet, ...
WILLY WIJAYA's user avatar
0 votes
1 answer
25 views

How can I learn and remove the linear trend in the residuals against the true response values generated by an ordinary neural network?

I built a neural network using PyTorch to predict y (a continuous variable) based on X consisting of m (=20) features. I found that the residuals (y_predicted – y_true) for the test data set show a ...
tunar's user avatar
  • 523
0 votes
0 answers
14 views

How to Handle Unknown Future Exogenous Variables in Seq2Seq Models for Time Series Forecasting? [closed]

Title: How to Handle Unknown Future Exogenous Variables in Seq2Seq Models for Time Series Forecasting? Question: I'm working on a sequence-to-sequence (seq2seq) model for predicting realized ...
DivertingPie's user avatar
1 vote
0 answers
19 views

Are these generated from my code the so called feature maps?

I assume that the way people build which activations detect specific pieces from an image is by executing the network and extracting the results at each layer; when the output is from a convolutional ...
Mah Neh's user avatar
  • 174
0 votes
0 answers
12 views

Why a project a reshape to 4x4x1024 for DCGAN?

In the paper Unsupervised Representation Learning with Deep Convolution Generative Adversarial Networks by Radford et. al. (2015), the model described projects and reshapes a 100 valued noise vector ...
Arjun V. Arun's user avatar
1 vote
0 answers
22 views

Why doesn't Kaiming/He weight Initialization seek a 50/50 compromise for forward and backward pass?

Sorry, please let me know if I'm off, but it seems that He initialization aims to either maintain a constant variance through the forward pass or through the backward pass. It seems the idea is that, ...
riley's user avatar
  • 11
1 vote
0 answers
11 views

Neural networks with uncertainties in training data

I have used Flax to train a neural network to fit a model to some data. All of the data points have a known uncertainty, as in each row has both a value and an uncertainty. (To be more explicit: the ...
rhombidodecahedron's user avatar
0 votes
0 answers
17 views

derivative of Logistic Regression with sigmoid func [duplicate]

I am having difficulty figuring out, why I get different answer from the professor. we are tasked with finding the deriative of the logistic regression cost function with the sigmoid function: $$L(w│D)...
Ofek nourian's user avatar
-1 votes
0 answers
57 views

How to make triplet loss more separable?

I use triplet learning and triplet loss to learn embeddings from images to tell at inference if two face pictures are same or different person (my real problem is different but it cannot be disclosed ...
Przemek B's user avatar
2 votes
0 answers
49 views

derivative of Logistic Regression (sigmoid) [closed]

I am having difficulty figuring out, why I get different answer from the professor. we are tasked with finding the deriative of the logistic regression cost function with the sigmoid function: $$ L(w│...
Ofek nourian's user avatar
1 vote
1 answer
30 views

(Multivariate) anomaly detection of (redundant) sensor data

I’m currently working on my master thesis and I’m looking for some inputs for the following situation: I have data of 2-20 sensors all measuring the same variable at 1-3 different locations in 15mins-...
Alexander's user avatar
0 votes
0 answers
28 views

Is Bootstrapping Independent Time Series to Construct Prediction Intervals Valid?

Question: I have a dataset consisting of multiple univariate time series, each representing an independent sequence of insurance claim amounts over time. My goal is to predict future claim amounts ...
Brandon_33's user avatar
1 vote
0 answers
19 views

Getting accurate Uncertainty from MFVI?

I wanted to know if there has been any research on methods to improve the accuracy of Mean-Field Variantional Inference (which doesn't discard the mean-field approximation). Apparently it is known to ...
profPlum's user avatar
  • 391
0 votes
0 answers
21 views

Wide Shallow Neural Networks VS Deep NN

I have several key points of understanding, but I cannot reach a final conclusion on why shallow neural networks cannot model data as effectively as deep neural networks. I understand that we can ...
rando's user avatar
  • 308
0 votes
0 answers
16 views

Why Contrastive Learning is needed?

Contrastive learning is a self-supervised, task-independent deep learning technique that allows a model to learn about data, even without labels. I am wondering one question why and how it is better ...
Rma's user avatar
  • 101
0 votes
0 answers
25 views

Why Do AR-NN Models Have Tighter Confidence Intervals Compared to Linear AR Models?

I have conducted a forecast for the following data series using different autoregressive models: Intercept-only, AR1, AR2, ARIMA BIC, ARIMA AIC, and AR-NN. Using the point forecasts, the AR1 model is ...
george1994's user avatar
0 votes
0 answers
13 views

How does highly imbalanced test data in certain splits of k-fold time-series cross-validation affect model performance?

I am working on a time-series classification (TSC) problem using k-fold time-series cross-validation (TSCV) to evaluate the performance of my models. My training data for each split is fairly balanced,...
Tov Nephesh's user avatar
0 votes
0 answers
17 views

Why is the gradient of $S = X \cdot W + b$, $dS$, equal to $\frac{\text{probs} - 1} {N}$, where $\text{probs}$ is the softmax classifier of $S$?

Why is the gradient of $S = X \cdot W + b$, $dS$, equal to $\frac{\text{probs} - 1} {N}$? Here, probs is a $1 \times K$ array which represents the probabilities that $X_i$ belongs to class $k$, where $...
Yash Jain's user avatar
  • 101
0 votes
0 answers
19 views

Why is my accuracy fluctuating for a while and then stuck? [duplicate]

I am building a cnn classifying model to predict images over 3 classes. The data is balanced, with 10.5k images for train ( 3.5k for each ), 3k validation images ( ...
Dragos123's user avatar
4 votes
1 answer
116 views

Choosing Between Intercept-Only and AR-NN Models: Justified to not use the model with the lowest RMSE/MAE?

I have created two autoregressive models for forecasting: a basic intercept-only model and an AR-NN (autoregressive neural network) model. Both models show similar performance based on recursive one-...
george1994's user avatar
3 votes
1 answer
69 views

What probability distribution is learned in this specific case? [duplicate]

I keep reading papers and blogposts where the training of a neural network is defined as learning some underlying probability distribution of the data. Imagine that you write CNN that outputs whether ...
Mah Neh's user avatar
  • 174
1 vote
0 answers
11 views

Can normalizing flows approximate bounded distributions in deep learning?

I’m exploring the use of normalizing flows in deep learning for generative modeling and I have a specific requirement: my target distributions are bounded (for example, between 0 and 1). I understand ...
Felipe Vieira's user avatar
2 votes
1 answer
39 views

VAEs: Why do we need the encoder for image generation?

I'm probably missing something obvious, but if we're only looking to generate images and are not interested in the latent space, why do we even need the encoder in VAEs? In my understanding, the ...
Jannik's user avatar
  • 125
0 votes
0 answers
14 views

ShapeNet VAE KL Divergence issues

I am trying to train a VAE on shapenet but I can't seem to make it work. Any help or ideas would be highly appreciated. Now the problem is whenever I apply the KL divergence loss the network seems to ...
Youssef's user avatar
2 votes
1 answer
32 views

Proper usage of K-fold cross validation and finalizing model

I am trying to learn about k-fold cross validation. I am using it on Kaggle dataset of brain tumors MRI trying to classify the images. Kaggle provides two directories Training with 5712 images and ...
Daniel11's user avatar
0 votes
0 answers
18 views

Understanding the Log-Sum-Exp (LSE) Operator in Chinchilla Replication Study, is e a set or exp?

I'm trying to understand the use of the Log-Sum-Exp (LSE) operator in the Chinchilla replication study, specifically the one referenced in the paper "Chinchilla Scaling: A replication attempt&...
Charlie Parker's user avatar
0 votes
0 answers
22 views

Why are the prediction intervals for my DNN regression model horizontal lines?

I am working on developing prediction intervals for deep reinforcement learning. Therefore, I am following the instructions given over here. I ran a small example using a simple deep-learning model to ...
desert_ranger's user avatar
0 votes
0 answers
15 views

Differentiable voting loss

I have a problem which needs me to assign a class to each object in a scene (say 0 or 1) represented by an image. I am posing this as a segmentation problem (since there are many objects in a scene ...
Abhijat Biswas's user avatar
0 votes
0 answers
16 views

Is it a good idea to incorporate a feature into the loss function when training a neural network model that does regression?

When training a neural network that does regression, assuming I have 3 features called "a", "b", and "c". The corresponding target is called "d". In theory, ...
sensationti's user avatar
0 votes
0 answers
29 views

Why reverse diffusion process is not a gaussian distribution?

The forward diffusion process, which goes from x_t to x_{t+1} is Gaussian, which is very reasonable as we go the next state by adding random gaussian noise. However, I do not understand why the ...
levitatmas's user avatar

1
2 3 4 5
162