Questions tagged [neural-networks]
Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.
8,071
questions
0
votes
1
answer
12
views
Why multiple linear regression perform better than single layer neural network in predicting time?
I am doing research on predicting failing time of a component of a machine.
Response is failing time of the component of a machine, and the input is location information (consists of integers).
I fit ...
- 1
1
vote
0
answers
4
views
Classification of intervals in time series data of multiple instances
I have a problem that I am trying to frame. I have signal data from ECG (a classic signal over time data). A close example here: https://github.com/jjongjjong/ECG_segmentation_1DUnet
I am basically ...
- 11
-2
votes
0
answers
20
views
What does the meaning of [[1x1,64],[3x3,64],[1x1,256]] in Resnet50? [duplicate]
In ResNet50,the first Bottleneck(second convolution layer) contain the kernel [[1x1,64],[3x3,64],[1x1,256]].
And I don't really understand the meaning of this combination.
0
votes
1
answer
26
views
using MSE loss paired with F-score in a classification model
for a video summarization project i use the features of each frame as input to predict if some of these frames are included in the summary or not.
one of the famous implementations i found had treated ...
0
votes
0
answers
34
views
Is it bad not to standardize all features (regression)?
I'm working with a neural network with two hidden layers for a regression task. My output values for the training set vary from 0 to 2000 and for the test set from 0 to 600. My main problem is ...
- 1
0
votes
0
answers
4
views
Do I need model calibration/feature normalization for reliable path integrated gradient/sampled Shapley feature attribution in a dnn model?
Are model calibration and feature normalization required for path integrated gradient and sampled Shapley-based feature attribution analyses to work properly in a deep neural network model?
I read ...
- 1
0
votes
0
answers
8
views
What is the Pseudo dimension for Lipschitz function class?
I am trying to get a bound on covering numbers with $L_2(P_n)$ norm using pseudo-dimension or fat-shattering numbers for Lipschitz classes.
-1
votes
0
answers
16
views
Must a CNN (both 1D and 2D) take input of the same size? [closed]
I have the notion that CNN input data must always be of the same dimensions. If we are feeding 1D tabular data, columns must be of the same numbers; if we are feeding 2D image data, all the images ...
- 2,098
0
votes
0
answers
17
views
Inaccurate estimates with approximate Bayesian computation using the abc package in R
I am trying to use approximate Bayesian computation (ABC) to calibrate parameters in order to obtain a model output variable that is close to an observed value in the literature. To do this, I am ...
0
votes
0
answers
6
views
How to apply CAMs on small images
I’m implementing the GradCam algorithm on several architectures, mainly Resnets. The main issue is that the processed data becomes very small in the last block, precisely the last feature map is 1x1.
...
- 1
3
votes
1
answer
34
views
Generating synthetic data with multiple records per ID
I would like to generate a synthetic dataset where there are multiple records per ID, and self-consistency is maintained among records of each ID.
For example, imagine a dataset where the ID is a ...
- 71
2
votes
0
answers
15
views
Example orthonormal basis of Word Embedding Space?
Models such as Word2Vec supposedly provide a bijection between language tokens and some "latent-space" that is in fact a high-dimensional vector space.
If this is a vector space, it should ...
- 21
1
vote
0
answers
17
views
Test loss immediately goes up on LSTM
I'm trying to create an LSTM that predicts the sixth sports match for team A based on a sequence of 5 previous matches. My data is set up in a structure like this. Team A game 1 vs random team, team B ...
- 11
0
votes
0
answers
16
views
Error term in SGD with momentum
I am reading the article "How Momentum really works" (https://distill.pub/2017/momentum/), and i am confused in one point:
I am trying to derive the convergence rate for momentum from the ...
- 121
0
votes
0
answers
30
views
Forecasting RNN and LSTM without X_test
Dear StackExchange Community,
My data is composed of only 1 time series variable (Stock prices of an asset)
I have splitted it to train and test subsets.
I have tarined an RNN and LSTM models with ...
1
vote
0
answers
19
views
How can I combine/pool the results of regression ANN? [closed]
In my analysis, the data contains 5 imputed dependent variables. So, after analyzing all of the dependent variables separately with a regression neural network, I need to combine/pool the results. ...
- 11
1
vote
0
answers
13
views
Impact of Pixel Normalization Technique on Weights, Gradients, and Activations in Neural Network
There are different ways to process an image either before or during the training of a neural network trained to take in image inputs.
Some of the pixel adjustment techniques used:
Scaling each pixel ...
- 11
1
vote
0
answers
33
views
How should you split up data in a train-test-validation split
I've seen it is generally recommended when using a train-test-validation data split, to first split your data into train and test datasets, and then furtherly split the train dataset into a train and ...
- 11
0
votes
1
answer
30
views
Relationship Between Neural Network Distances and Performance
I've been wondering whether there might be a correlation between the "distance" among neural network weights and their performance.
To elaborate, consider the following scenario:
We have ...
1
vote
0
answers
40
views
Which classic ML algorithms can be written as neural networks?
Many classic machine learning algorithms can be reframed as simple neural networks.
For instance:
Linear regression can be thought of as a neural network with one linear layer with $p$ inputs and $1$ ...
- 515
0
votes
1
answer
25
views
Question on the Partial Derivative of the Cross-Entropy Loss in SGD for Neural Networks
I'm currently learning about neural networks and stumbled upon a confusion related to the use of Stochastic Gradient Descent (SGD) in training. Specifically, I'm puzzled about the computation of the ...
John Title
0
votes
0
answers
22
views
Large Scale Missing Data & Imputation of Time Series Data in Neural Networks [duplicate]
I know there has already been a lot of discussion about this topic, but I have reasons to believe it still remains unanswered and lacks several justifications.
Suppose we have an time series feature ...
- 101
1
vote
0
answers
27
views
Hinge loss vs categorical_hinge loss in Keras [closed]
I was working on a project where I implemented a CNN model with hinge loss in Keras. The task is a multiclass classification where I try to classify sensor measurements into 6 classes. I'm aware that ...
0
votes
0
answers
12
views
Normalization of time-series data with time-varying variance
I'm building a neural network (CNN) model for a regression problem with time-series data. Both input and output are multi-variate zero-mean timeseries data with time-varying variance. Currently, I am ...
- 111
0
votes
0
answers
19
views
Class activation map to weakly supervised semantic segmentation
I have a classifier model using the fine-tuning technique and training it with a dataset for the classification of tree species, achieving a 98% F1 score from sklearn in training like this image. I'm ...
- 1
1
vote
1
answer
36
views
results of a regression predictor
I have a neural network trained to predict values from timeseries.
the target (which is hopefully to be predicted by NN) is always in range 0.0~1.0, and has these statistic features:
...
- 13
0
votes
0
answers
19
views
why Conditional VAE require conditioning of the encoder
looking at this blogpost and in many other, the cVAE looks like this:
Now, my question is... why do we need the label on the encoder level? Clearly that information is already inside the image, thus ...
- 1,139
0
votes
0
answers
4
views
Custom Model For Approximating Sin Function Using Backpropagation [duplicate]
I have very simple custom model which I am doing experiment with, I have model which takes one input and produce one output. the model equation is: y = sin(ax + b). (a) and (b) are single learnable ...
- 1
0
votes
1
answer
8
views
Should the testing data be uniquely distinct and come from different source/dataset than the training data?
I am building an audio classification system using CNN. My dataset consists of different audio I have recorded and spliced to equal time lengths. Like with any other common ML or DL tasks, I am to ...
- 1
0
votes
0
answers
110
views
Transformer with just one input vector
I have a problem where I am mapping from 1D input sequences of length L to 1D output sequences, also of length L. These sequences contain numerical data. The input sequence is the time evolution of a ...
0
votes
0
answers
13
views
How to perform random walk on multilayer network to predict new edges
I have a multi-layer network that is a union of 3 networks (field of human biology/ Omics data). The 3 networks have dense connections within each other (local), however sparse connections to each ...
- 21
0
votes
0
answers
16
views
Derivatives in Computation graph
In Advanced Learning Algorithms by Andrew Ng, Coursera:
One thing that makes backprop efficient is you
notice that when we do the right-to-left calculation,
we had to compute this term,
the ...
- 101
1
vote
1
answer
64
views
How to analyze uncorrelated data?
Sometimes we encounter data that is uncorrelated. Specifically, from the correlation matrix we observe that the target variable shows low or no correlation with any of the features.
To provide context,...
3
votes
1
answer
43
views
Binary decision boundary requiring 2 hidden layers in neural network with limited neurons
I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer (where number of neurons ...
- 131
0
votes
1
answer
19
views
Does a model learn the same attention scores when retrained?
As in the title, should I expect a model to learn almost the same attention scores in its attention layers when I train it? Perhaps only in the first one if there are multiple such layers?
It feels ...
- 11
0
votes
0
answers
14
views
Why my numpy Neural Network doesnt converge when the loss is "Mean Squared Error "? [closed]
I build a MLP with numpy to approximate the cosine function, when the loss is Mean error it converge:
...
Gaston Lagaff
0
votes
0
answers
11
views
Vision transformer overfitting, cannot figure the reason why as many experiments
I am training imaging data with ~1000 channels on a modified vision transformer model.
Preprocessing
I am limited in the number of samples as I only have 10 images (~200x200x1000) available to me ...
0
votes
0
answers
16
views
Machine Learning Experiment: should training parameters be fixed for a valid comparisons across model?
I am training an autoencoder three times, each time on a different dataset. The three datasets all have the same number of features, but have vastly different sizes.
Assuming one of the datasets is A ...
0
votes
0
answers
14
views
Neural net and category separation with interaction terms
We have a stream of tabular data consiting of categorical and numerical features. One category is somehow crucial in either affecting the target, interacting with other features and sorting the data ...
- 31
0
votes
0
answers
16
views
Extremely Small Output Weight Values in Echo State Network
I have an echo state network that is producing an output weight matrix with extremely small output weights (on the order of 10^-200). Ideally, these weights should be within a more reasonable interval,...
0
votes
0
answers
6
views
Two different senarios to train only the last layer of a convolutional neural network
I have a convolutional neural network that has already been trained on data A. The network consists of feature extraction layers and two classification layers. I have trained only it's last ...
0
votes
0
answers
38
views
Should transfer learning from the whole data set be used tuning individual model?
I have a dataset of many timeseries and I want to develop anomaly detection with neural networks for each individual timeseries.
Are there any benefits training a generic model on the whole dataset ...
- 31
0
votes
0
answers
20
views
Fluctuating Validation Loss & Accuracy during Transfer Learning (ResNet50) - FER+ Dataset
I'm trying to build a CNN model for image classification, more specifically emotion classification using the FER+ dataset which is proving difficult to work with.
I've tried several variations of ...
0
votes
0
answers
19
views
How to Plot and Interpret the ROC Curve for Segmentation-based Object Detection Models?
I'm trying to plot the ROC Curve for a number of target/object detection models and compare their performance. The pre-trained models in question take an input image and they output a mask image where ...
- 1
0
votes
0
answers
10
views
Ensuring numerical stability of probabilities in identifiable multinomial regression
I'm looking to understand the differences between the functions for probabilities induced in an identified multinomial GLM, and the softmax function commonly used in classification for neural networks....
- 1
0
votes
0
answers
22
views
How to modify my classification neural net with keras to improve accuracy? [duplicate]
I am trying to build a neural net to predict binary output [0,1]. I have a pretty small dataset 600 samples, 200 of them label 0, 400 of them label 1. I have 23 features, some of them are ...
0
votes
2
answers
62
views
simple ANN as a set of linear transformations
We cannot classify the points of the XOR problem with a single perceptron in the hidden layer. However, we can achieve this by using two perceptrons in the hidden layer and one for the output layer, ...
- 1
4
votes
2
answers
99
views
Scaling laws for neural network memorization
I would like to ask a generalization of this question: How to perfectly overfit neural network to memorize input?
Are there any scaling laws for neural network memorization? In other words, if I have ...
2
votes
3
answers
77
views
Are reinforcement learning and deep learning equivalent?
Can a deep learning classifier, trained on a dataset derived from a reinforcement learning (RL) agent's interactions with an environment, achieve the same performance as the RL agent itself? Assuming ...
- 129
2
votes
0
answers
71
views
Do discontinuous functions have subgradients also?
Typically, the subgradient is defined for convex functions. And convex functions are continuous.
However, DeepMind's VQ-VAE paper defines a model with a discontinuous vector quantization (VQ) layer, ...
- 1,331