Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

How to use cosine similarity within triplet loss

The triplet loss is defined as follows: $$ L(A, P, N) = max(‖f(A) - f(P)‖² - ‖f(A) - f(N)‖² + margin, 0) $$ where $A$=anchor, $P$=positive, and $N$=negative are the data samples in the loss, and $...
user avatar
  • 721
0 votes
0 answers
10 views

Restricted Boltzmann Machine derivation of conditional independencies

In restricted boltzmann machines the formula $P(H|V)=\prod_{j}P(H_j|V)$ holds. One can derive the formula the following way: $P(H=h|V=v)=\frac{P(V=v,H=h)}{P(V=v)}=\frac{P(V=v,H=h)}{\sum_{H}P(H=h,V=v)}...
user avatar
0 votes
0 answers
7 views

How can I evaluate my work if there is no benchmark study for my targeted domain?

I have some questions if possible... My project is to create a model to detect fake news in a specific domain, which has not been investigated in this specific domain by previous studies. Data on this ...
user avatar
1 vote
1 answer
20 views

How to model distance and interpret predictors

I have a set of 2 - 10 explanatory variables which I'd like to use to predict the response variable, distance. The explanatory variables describe the flight of a projectile (velocity, spin, angle) and ...
user avatar
  • 167
1 vote
0 answers
7 views

DQN network cannot distinguish output ports with different actions

I am trying to implement a Deep Q-Network (DQN) with a prediction network and a target network. Both nets take the state as input and have two output ports each corresponding to a different action. ...
user avatar
0 votes
0 answers
5 views

Optimal number of steps per epoch for maximum accuracy on neural networks

This question has a very good answer discussing optimal mini-batch size for training neural networks, that points out that the final accuracy of a model usually decreases when using very large batches ...
user avatar
2 votes
0 answers
31 views

Natural gradients with Moore–Penrose inverse of the Fisher information matrix

I'd like to show you my rough sketch for scaling up natural gradients to deep neural networks that appears to be easy to automate just like automatic differentiation. I think there must be a flaw ...
user avatar
1 vote
0 answers
48 views

Poor reconstructions from using sigmoid in last layer of variational autoencoder

I have trained a variational autoencoder (VAE) using Pytorch Lightning to reproduce images. Without sigmoid, reproductions are good. However, some output image ...
user avatar
  • 23
0 votes
0 answers
27 views

How to develop Multistage classification model using deep learning

I am little confused while doing multistage classification using deep learning model. I have data as below: ...
user avatar
1 vote
1 answer
12 views

Reinforcement Learning - what causes a plateau in the performance of the model during training?

I am working on a Reinforcement Learning problem, but since the underlying model is a neural network, I think this might have similarities to a supervised learning problem. Below is a screenshot of my ...
user avatar
0 votes
1 answer
19 views

What do the "in/out features" of a neural network mean?

I am working on a Reinforcement Learning problem in StableBaselines3, but I don't think that really matters for this question. SB3 is based on PyTorch. So, below is a screenshot of my model ...
user avatar
0 votes
0 answers
6 views

Data split when using pseudo-labeling semi-supervised learning method

I'm trying to train a 3D segmentation model. The dataset I own consists of small number of labeled samples(~21) and a lot of unlabeled samples(~200). I'm using a simple semi-supervised method, where I'...
user avatar
1 vote
0 answers
17 views

Derivative of neural network respect to input

I have a neural network like this $x=\text{input}$ $z_1=W_1\cdot x+b_1$ $h_1=\text{relu}(z_1)$ $z_2=W_2\cdot h_1+W_{1x}\cdot x+b_2$ $h_2=\text{relu}(z_2)$ $y=W_3\cdot h_2+W_{2x}\cdot x+b_3$ input and ...
user avatar
1 vote
0 answers
13 views

Is there literature about how different neural network layers recognise certain features?

Let's consider a deep convolutional network. It seems that there is some consensus on the following notions: 1. Shallow layers tend to recognise more low-level features such as edges and curves 2. ...
user avatar
  • 649
0 votes
0 answers
20 views

Model the style (not content) of fonts

I'm looking at creating a model to group similar fonts together. I'm not interested in the content of the fonts (e.g. the words), just the style (thickness of letters, spacing, etc.). My initial idea ...
user avatar
0 votes
0 answers
14 views

Calculating approximate probability with neural network

Many times in the literature we say that neural network approximate a distribution, either conditional (for example for discriminative networks, $p_{\theta}(y|x)$) or not (for example in generative ...
user avatar
0 votes
0 answers
31 views

Machine Learning for Stock Price Prediction Issue

In financial application, someone might make use of machine learning techniques in stock price prediction, e.g. LSTM. In general, before training the model, in light of the model robustness, some ...
user avatar
0 votes
0 answers
12 views

How to model a data for an extrapolation/regression problem - traditional ANN - Multi Layer Perceptron [closed]

Telecom dataset shows only 6 parameters such as Day, Start Time, End Time, Base Station Location, User ID. The trajectory of users can be found by 6 month dataset. My goal is to estimate the ...
user avatar
1 vote
1 answer
21 views

Where can I find pre-trained fully convolutional neural networks? [closed]

I know that fully convolutional neural networks can be used for classifying images of arbitrary sizes. I would like to use some pre-trained fully-convolution neural networks for extracting features in ...
0 votes
0 answers
13 views

F1 score for LSTM-RNN [migrated]

I created a LSTM-RNN model for binary classification. I want to calculate F-1 score for this model but the code I used to calculate the F-1 score for ANN does not work for RNN. How can I measure F-1 ...
user avatar
  • 1
0 votes
0 answers
5 views

Stochastic objective function in Adam [duplicate]

Relating to the question What is a stochastic objective function?. In the paper of the Adam Algorithm they also mention a Stochastic objective function when explaining the algorithm. In this context I ...
user avatar
1 vote
1 answer
15 views

Should I train my classifier with examples that are outside my classes of interest? And should I create an "others" class to handle them?

This is a 2 part question regarding a multi-class classifier based on a neural network that is expected to predict whether the input image has a cat or a dog. If shown something different (like a man),...
user avatar
2 votes
0 answers
55 views

Predict uncertainty in NN prediction [closed]

Quick question related to maximizing the likelihood as loss function for NN. In case of regression, by assumption we have that the data is composed by Gaussian noise... however, the wider the noise, ...
user avatar
-2 votes
0 answers
22 views

understanding poor performance of a neural network in R [duplicate]

(This question was closed as a duplicate to this post, but I'm not observing a situation where a net performs poorly on test data, I'm describing a net performing poorly on completely new data ...
user avatar
  • 450
-2 votes
0 answers
29 views

self study part deux: why is my neural net so bad on new data? [duplicate]

Follow up to a previous question. Say I have this data: ...
user avatar
  • 450
1 vote
0 answers
12 views

"Biasing" a generalized trained object detector towards specific examples during inference

I have trained an object detection deep learning model on many different types of cars (shape, color, car model variations etc.). I'm just using a single class "Car" for all the different ...
user avatar
  • 11
0 votes
0 answers
10 views

Detect anomalies in delivery dates using a neural network

I have a dataset with the following datetime values of part delivery orders: OrderId RequestedDeliveryDate ActualDeliveryDate OrderCreationDate OrderChangeDate PartReplenishmentTime 1 2001-03-09 ...
user avatar
  • 1
10 votes
1 answer
1k views

self study: why is my neural network so much worse than my random forest

Playing around with machine learning in R. Say I have this arbitrary function: ...
user avatar
  • 450
1 vote
1 answer
20 views

Mismatch between the dimensions of Jacobian matrixes when calculating derivatives during backprop?

I am trying to understand how back propagation works for a linear layer using minibatches by following this post: https://web.eecs.umich.edu/~justincj/teaching/eecs442/notes/linear-backprop.html. ...
user avatar
  • 133
0 votes
0 answers
6 views

Am I understanding training a model to use in a similar task (both translations but from different language pairs) later, wrong?

I am currently training an mt5 in Spanish to English (and vice versa) translation. That works with a bleu of 40ish. (both ways). I then want to use that same model I trained to improve the translation ...
user avatar
1 vote
0 answers
7 views

CT-Scan Classification Overfitting Problem

I am currently trying to train pretrained convolutional neural networks trained on the imagenet dataset to be able to classify ct-scans into two classes. Viral Pneumonia and Normal. I am using K-fold ...
user avatar
0 votes
0 answers
8 views

Image generation with multiple images as inputs

I am fairly new to machine learning. I am trying to generate a new image from other images of the same shape. An example of the image I'm trying to generate is an Hi-C data matrix: https://encrypted-...
user avatar
1 vote
1 answer
24 views

How to design a neural network?

For my research project, I am expected to propose a MLP for solving a disease classification problem. Whenever i've done this before, it's been a lot of trial and error/ adding and removing layers etc,...
user avatar
1 vote
1 answer
29 views

What are "volatile" learning curves indicative of? [duplicate]

I have a dataset set with ~40 features onto which I'm applying a multi-layer perceptron for regression purposes. The train, validation, and test sets are made up of 3M, 800K, and 800K examples each, ...
user avatar
  • 565
1 vote
0 answers
25 views

Is there an empirical rule for selecting the value of label smoothing?

I am wondering if there is any emperical rule for selecting the value of label smoothing when training a neural network. Let's define smoothed prediction targets in relation to a value $\epsilon$ to ...
user avatar
0 votes
0 answers
10 views

High resolution images as VGG input

In the VGG paper, it is explained that input images are randomly cropped to 224x224 from rescaled images. I feel that input data of say, a resolution up to 512x384, would be appropriately augmented ...
user avatar
  • 649
0 votes
0 answers
22 views

At what point in the ML pipeline should I under/over sample?

I have an imbalanced multi-class dataset, and am under/over sampling to balance it out. My questions is when should I do this resampling? Should it occur before creating the test set, before creating ...
user avatar
  • 153
1 vote
0 answers
13 views

How does the full derivative of softmax + cross entropy have the correct dimensions?

The blog post the softmax function and its derivative explains the following: Imagine that each input has $N$ features / pixels / etc. Imagine each input can be classified into $C$ classes Let the ...
user avatar
  • 133
3 votes
1 answer
277 views

Does the n >> p holds also for minibatches

It's pretty known that when dealing with models (without regularization) the main assumption is $n >> p$ where $p$ is the number of features in the dataset Let's suppose that we have 1.000.000 ...
user avatar
1 vote
1 answer
23 views

Splitting medical dataset by patient

I am currently trying to train a CNN model to classify CT-scans. I split the dataset using K-fold cross-validation and since the dataset I am using contains multiple slices per patient, I split the ...
user avatar
2 votes
0 answers
28 views

Is normal distribution better than skewed distribution for machine learning input features? [closed]

Distribution of a particular feature in my ML dataset is skewed as shown below. Log of this feature looks like a normal distribution. Can the latter distribution offer better predictability in a ...
user avatar
0 votes
0 answers
11 views

How to restore Optuna's finished study from logs?

I wish to restore optuna.study.Study class object from uncomplete (5000 last lines, which is enough for my goals) logs. They look like that: ...
user avatar
0 votes
0 answers
17 views

What is L1 and L2 regularization? [closed]

When to use L1 regularization, when to use L2 regularization. Why do in some cases L1 work better and in some cases L2 work better? What is the mathematical reasoning behind this?
user avatar
0 votes
0 answers
10 views

Assemble neural networks to improve performance

I am approaching the world of Geometric Deep Learning for the first time and I have a question, I hope someone can answer it. I am currently working on models to classify some drugs as highly active, ...
user avatar
0 votes
0 answers
7 views

Why are undirected graphical models (MRFs) not represented directly in terms of probability like directed graph models?

I have been reading the Deep Learning Book by Ian Goodfellow, and in that, there is a discussion about graphical models like Bayesian belief networks and Markov Random Fields. Here: One key difference ...
user avatar
1 vote
1 answer
18 views

How does randomized search cv algorithm work?

I am building a DNN, and I used Randomizedsearchcv from Scikit learn to optimise the hyper-parameters. Hence, I have one question about this: As I understood, the basic of random search is to try out ...
user avatar
  • 11
0 votes
0 answers
17 views

Imbalanced classification model training

Consider a dataset with 11 variables where 1 variable Y is considered the label having binary values of 0 and 1. Consider also that in a dataset like this the 0s are far more than the 1s( i.e. 9000 ...
user avatar
1 vote
0 answers
11 views

State-of-the-art techniques for regularizing Neural Networks?

For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's. Have any significant/strong competitors risen up since then?
user avatar
  • 389
0 votes
0 answers
12 views

When is RMSprop Gradient Descent better than Momentum Gradient Descent

Line of thought The weight updates in RMSprop gradient descent can be considered as normalizing a gradient $d\Theta_i$ by the factor $1/(\sqrt{Sd\Theta_i} + \epsilon)$. If $\nabla_\Theta J$ of the ...
user avatar
  • 83
0 votes
0 answers
23 views

Why does my auto-encoder with batch norm produce dramatically different images in train and evaluate mode?

I am training an AutoEncoder on some cats and dogs images. After training the model, I get reasonable reconstruction results. So I save the model. (The model only has converted layers and BN layers). ...
user avatar

1
2 3 4 5
188