Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
18 views

What does the meaning of [[1x1,64],[3x3,64],[1x1,256]] in Resnet50? [duplicate]

In ResNet50,the first Bottleneck(second convolution layer) contain the kernel [[1x1,64],[3x3,64],[1x1,256]]. And I don't really understand the meaning of this combination.
Yajou Wang's user avatar
0 votes
1 answer
26 views

using MSE loss paired with F-score in a classification model

for a video summarization project i use the features of each frame as input to predict if some of these frames are included in the summary or not. one of the famous implementations i found had treated ...
moha tech's user avatar
0 votes
0 answers
33 views

Is it bad not to standardize all features (regression)?

I'm working with a neural network with two hidden layers for a regression task. My output values for the training set vary from 0 to 2000 and for the test set from 0 to 600. My main problem is ...
stella's user avatar
  • 1
0 votes
0 answers
3 views

Do I need model calibration/feature normalization for reliable path integrated gradient/sampled Shapley feature attribution in a dnn model?

Are model calibration and feature normalization required for path integrated gradient and sampled Shapley-based feature attribution analyses to work properly in a deep neural network model? I read ...
jjwest's user avatar
  • 1
0 votes
0 answers
8 views

What is the Pseudo dimension for Lipschitz function class?

I am trying to get a bound on covering numbers with $L_2(P_n)$ norm using pseudo-dimension or fat-shattering numbers for Lipschitz classes.
fun yaaar's user avatar
-1 votes
0 answers
16 views

Must a CNN (both 1D and 2D) take input of the same size? [closed]

I have the notion that CNN input data must always be of the same dimensions. If we are feeding 1D tabular data, columns must be of the same numbers; if we are feeding 2D image data, all the images ...
user366312's user avatar
  • 2,088
0 votes
0 answers
17 views

Inaccurate estimates with approximate Bayesian computation using the abc package in R

I am trying to use approximate Bayesian computation (ABC) to calibrate parameters in order to obtain a model output variable that is close to an observed value in the literature. To do this, I am ...
Paul Racino's user avatar
0 votes
0 answers
6 views

How to apply CAMs on small images

I’m implementing the GradCam algorithm on several architectures, mainly Resnets. The main issue is that the processed data becomes very small in the last block, precisely the last feature map is 1x1. ...
PiEmmeC's user avatar
3 votes
1 answer
34 views

Generating synthetic data with multiple records per ID

I would like to generate a synthetic dataset where there are multiple records per ID, and self-consistency is maintained among records of each ID. For example, imagine a dataset where the ID is a ...
user12138762's user avatar
2 votes
0 answers
15 views

Example orthonormal basis of Word Embedding Space?

Models such as Word2Vec supposedly provide a bijection between language tokens and some "latent-space" that is in fact a high-dimensional vector space. If this is a vector space, it should ...
Vector's user avatar
  • 21
1 vote
0 answers
16 views

Test loss immediately goes up on LSTM

I'm trying to create an LSTM that predicts the sixth sports match for team A based on a sequence of 5 previous matches. My data is set up in a structure like this. Team A game 1 vs random team, team B ...
WILLYB's user avatar
  • 11
0 votes
0 answers
16 views

Error term in SGD with momentum

I am reading the article "How Momentum really works" (https://distill.pub/2017/momentum/), and i am confused in one point: I am trying to derive the convergence rate for momentum from the ...
Patricio's user avatar
  • 121
0 votes
0 answers
30 views

Forecasting RNN and LSTM without X_test

Dear StackExchange Community, My data is composed of only 1 time series variable (Stock prices of an asset) I have splitted it to train and test subsets. I have tarined an RNN and LSTM models with ...
Amirgiano's user avatar
1 vote
0 answers
19 views

How can I combine/pool the results of regression ANN? [closed]

In my analysis, the data contains 5 imputed dependent variables. So, after analyzing all of the dependent variables separately with a regression neural network, I need to combine/pool the results. ...
minre's user avatar
  • 11
1 vote
0 answers
13 views

Impact of Pixel Normalization Technique on Weights, Gradients, and Activations in Neural Network

There are different ways to process an image either before or during the training of a neural network trained to take in image inputs. Some of the pixel adjustment techniques used: Scaling each pixel ...
Kinshu's user avatar
  • 11
1 vote
0 answers
33 views

How should you split up data in a train-test-validation split

I've seen it is generally recommended when using a train-test-validation data split, to first split your data into train and test datasets, and then furtherly split the train dataset into a train and ...
sammcm998's user avatar
0 votes
1 answer
30 views

Relationship Between Neural Network Distances and Performance

I've been wondering whether there might be a correlation between the "distance" among neural network weights and their performance. To elaborate, consider the following scenario: We have ...
nickromandini's user avatar
1 vote
0 answers
40 views

Which classic ML algorithms can be written as neural networks?

Many classic machine learning algorithms can be reframed as simple neural networks. For instance: Linear regression can be thought of as a neural network with one linear layer with $p$ inputs and $1$ ...
Steven Gubkin's user avatar
0 votes
1 answer
25 views

Question on the Partial Derivative of the Cross-Entropy Loss in SGD for Neural Networks

I'm currently learning about neural networks and stumbled upon a confusion related to the use of Stochastic Gradient Descent (SGD) in training. Specifically, I'm puzzled about the computation of the ...
user avatar
0 votes
0 answers
22 views

Large Scale Missing Data & Imputation of Time Series Data in Neural Networks [duplicate]

I know there has already been a lot of discussion about this topic, but I have reasons to believe it still remains unanswered and lacks several justifications. Suppose we have an time series feature ...
LazyAnalyst's user avatar
1 vote
0 answers
27 views

Hinge loss vs categorical_hinge loss in Keras [closed]

I was working on a project where I implemented a CNN model with hinge loss in Keras. The task is a multiclass classification where I try to classify sensor measurements into 6 classes. I'm aware that ...
Imene Charabi's user avatar
0 votes
0 answers
12 views

Normalization of time-series data with time-varying variance

I'm building a neural network (CNN) model for a regression problem with time-series data. Both input and output are multi-variate zero-mean timeseries data with time-varying variance. Currently, I am ...
P. Leibner's user avatar
0 votes
0 answers
19 views

Class activation map to weakly supervised semantic segmentation

I have a classifier model using the fine-tuning technique and training it with a dataset for the classification of tree species, achieving a 98% F1 score from sklearn in training like this image. I'm ...
Min's user avatar
  • 1
1 vote
1 answer
36 views

results of a regression predictor

I have a neural network trained to predict values from timeseries. the target (which is hopefully to be predicted by NN) is always in range 0.0~1.0, and has these statistic features: ...
Bikay's user avatar
  • 13
0 votes
0 answers
19 views

why Conditional VAE require conditioning of the encoder

looking at this blogpost and in many other, the cVAE looks like this: Now, my question is... why do we need the label on the encoder level? Clearly that information is already inside the image, thus ...
Alberto's user avatar
  • 1,139
0 votes
0 answers
4 views

Custom Model For Approximating Sin Function Using Backpropagation [duplicate]

I have very simple custom model which I am doing experiment with, I have model which takes one input and produce one output. the model equation is: y = sin(ax + b). (a) and (b) are single learnable ...
mohammad's user avatar
0 votes
1 answer
8 views

Should the testing data be uniquely distinct and come from different source/dataset than the training data?

I am building an audio classification system using CNN. My dataset consists of different audio I have recorded and spliced to equal time lengths. Like with any other common ML or DL tasks, I am to ...
Flash's user avatar
  • 1
0 votes
0 answers
110 views

Transformer with just one input vector

I have a problem where I am mapping from 1D input sequences of length L to 1D output sequences, also of length L. These sequences contain numerical data. The input sequence is the time evolution of a ...
Blahblahblacksheep's user avatar
0 votes
0 answers
13 views

How to perform random walk on multilayer network to predict new edges

I have a multi-layer network that is a union of 3 networks (field of human biology/ Omics data). The 3 networks have dense connections within each other (local), however sparse connections to each ...
oelakad's user avatar
  • 21
0 votes
0 answers
16 views

Derivatives in Computation graph

In Advanced Learning Algorithms by Andrew Ng, Coursera: One thing that makes backprop efficient is you notice that when we do the right-to-left calculation, we had to compute this term, the ...
derivative's user avatar
1 vote
1 answer
64 views

How to analyze uncorrelated data?

Sometimes we encounter data that is uncorrelated. Specifically, from the correlation matrix we observe that the target variable shows low or no correlation with any of the features. To provide context,...
Avijit Jaiswal's user avatar
3 votes
1 answer
43 views

Binary decision boundary requiring 2 hidden layers in neural network with limited neurons

I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer (where number of neurons ...
Regina Dea's user avatar
0 votes
1 answer
19 views

Does a model learn the same attention scores when retrained?

As in the title, should I expect a model to learn almost the same attention scores in its attention layers when I train it? Perhaps only in the first one if there are multiple such layers? It feels ...
rick's user avatar
  • 11
0 votes
0 answers
14 views

Why my numpy Neural Network doesnt converge when the loss is "Mean Squared Error "? [closed]

I build a MLP with numpy to approximate the cosine function, when the loss is Mean error it converge: ...
user avatar
0 votes
0 answers
11 views

Vision transformer overfitting, cannot figure the reason why as many experiments

I am training imaging data with ~1000 channels on a modified vision transformer model. Preprocessing I am limited in the number of samples as I only have 10 images (~200x200x1000) available to me ...
ConfusedPotato's user avatar
0 votes
0 answers
16 views

Machine Learning Experiment: should training parameters be fixed for a valid comparisons across model?

I am training an autoencoder three times, each time on a different dataset. The three datasets all have the same number of features, but have vastly different sizes. Assuming one of the datasets is A ...
BovineScatologist's user avatar
0 votes
0 answers
14 views

Neural net and category separation with interaction terms

We have a stream of tabular data consiting of categorical and numerical features. One category is somehow crucial in either affecting the target, interacting with other features and sorting the data ...
funkymind's user avatar
0 votes
0 answers
16 views

Extremely Small Output Weight Values in Echo State Network

I have an echo state network that is producing an output weight matrix with extremely small output weights (on the order of 10^-200). Ideally, these weights should be within a more reasonable interval,...
Jonathan Frutschy's user avatar
0 votes
0 answers
6 views

Two different senarios to train only the last layer of a convolutional neural network

I have a convolutional neural network that has already been trained on data A. The network consists of feature extraction layers and two classification layers. I have trained only it's last ...
Nastaran Takmilhomayouni's user avatar
0 votes
0 answers
38 views

Should transfer learning from the whole data set be used tuning individual model?

I have a dataset of many timeseries and I want to develop anomaly detection with neural networks for each individual timeseries. Are there any benefits training a generic model on the whole dataset ...
Art's user avatar
  • 31
0 votes
0 answers
20 views

Fluctuating Validation Loss & Accuracy during Transfer Learning (ResNet50) - FER+ Dataset

I'm trying to build a CNN model for image classification, more specifically emotion classification using the FER+ dataset which is proving difficult to work with. I've tried several variations of ...
RoliPoliOli's user avatar
0 votes
0 answers
19 views

How to Plot and Interpret the ROC Curve for Segmentation-based Object Detection Models?

I'm trying to plot the ROC Curve for a number of target/object detection models and compare their performance. The pre-trained models in question take an input image and they output a mask image where ...
Tungdil's user avatar
0 votes
0 answers
10 views

Ensuring numerical stability of probabilities in identifiable multinomial regression

I'm looking to understand the differences between the functions for probabilities induced in an identified multinomial GLM, and the softmax function commonly used in classification for neural networks....
bill's user avatar
  • 1
0 votes
0 answers
22 views

How to modify my classification neural net with keras to improve accuracy? [duplicate]

I am trying to build a neural net to predict binary output [0,1]. I have a pretty small dataset 600 samples, 200 of them label 0, 400 of them label 1. I have 23 features, some of them are ...
Anna Svirshchevsky's user avatar
0 votes
2 answers
62 views

simple ANN as a set of linear transformations

We cannot classify the points of the XOR problem with a single perceptron in the hidden layer. However, we can achieve this by using two perceptrons in the hidden layer and one for the output layer, ...
Mag's user avatar
  • 1
4 votes
2 answers
99 views

Scaling laws for neural network memorization

I would like to ask a generalization of this question: How to perfectly overfit neural network to memorize input? Are there any scaling laws for neural network memorization? In other words, if I have ...
zfj3ub94rf576hc4eegm's user avatar
2 votes
3 answers
77 views

Are reinforcement learning and deep learning equivalent?

Can a deep learning classifier, trained on a dataset derived from a reinforcement learning (RL) agent's interactions with an environment, achieve the same performance as the RL agent itself? Assuming ...
yang's user avatar
  • 129
2 votes
0 answers
70 views

Do discontinuous functions have subgradients also?

Typically, the subgradient is defined for convex functions. And convex functions are continuous. However, DeepMind's VQ-VAE paper defines a model with a discontinuous vector quantization (VQ) layer, ...
MWB's user avatar
  • 1,333
1 vote
0 answers
29 views

Is using aggregated usage data more efficient than using flattened usage data to build a ML model in anomaly detection? [closed]

We're tracking users' hourly usage on our cloud service and have a risk model that uses aggregated usage data plus other signals to identify potential fraudsters. Basically, it's an anomaly detection ...
Jiayu Zhang's user avatar
0 votes
1 answer
17 views

A question about K-fold cross validation in the context of neural networks

Let's say that I want to perform K-fold cross validation for a neural network model. So, as usual, I split my dataset D into $K$ different folds. The fold $k$ is the validation fold while the ...
James Arten's user avatar

1
2 3 4 5
162