Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
1
vote
0answers
26 views

Does it make sense to do VLAD encoding on top of features extracted with CNNs?

I have a deep learning framework which extracts features of patches of my images and builds a dictionary with PCA and then k-Means. Then the framework detects anomalies based on the distance of the ...
0
votes
0answers
17 views

Can different types of feature scaling result in different prediction performance and how to choose one type?

Being new to machine learning and currently making use of a MLP-Classifier from scikit learn to solve a multi-class multi-label classification problem, I was wondering how to decide on a type of ...
1
vote
0answers
25 views

Is it possible to use a Variational Autoencoder as a Feature Extractor like a standard CNN?

I have looked at autoencoders as feature extractors and I wondered if a variational autoencoder could also be used in the same way. For standard autoencoders you would just use the encoder-network, ...
0
votes
0answers
10 views

Extract Features at Multiple Image-Scales

I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-...
0
votes
1answer
14 views

conceptually, how to use NLP to predict a numeric output

suppose I'm trying to use medical notes to predict the cost of medical service. For example, a patient will call in, tell the operator how they feel, their diagnosis, etc etc, and the operator will ...
0
votes
0answers
17 views

Spatial Pyramid Pooling vs Adaptive Max Pooling

I was confused about whether Spatial Pyramid Pooling and Adaptive Max Pooling are different layers or same layers with different name. Can anyone please mention how the 2 are different (if there is ...
0
votes
0answers
11 views

High precision, but low recall on training data. What could be wrong?

I am training RNN for timeseries binary classification. I observed that network has high precision, but low recall on both training and testing data. I tried multiple architectures, but same problem ...
6
votes
1answer
204 views

Is there a formal proof that Autoencoders perform non-linear PCA?

I have seen this statement in various blog posts, papers etc., and the claim is intuitive for me, however I had a hard time finding a paper with the actual proof for that. I guess one could view that ...
0
votes
0answers
16 views

Evaluating a Keypoint Regression CNN Model with Euclidean Distances

I have trained an encoder-decoder keypoint regression model that outputs 2d heatmaps using MSE loss on a dataset with 14300 training images and 3700 test images. The encoder is a ResNet-50 that was ...
0
votes
0answers
11 views

Object detection vs segmentation?

My problem statement is as follows: "Object detection is the concept of classifiying & localising an object in an image, and semantic segmentation is the concept of labeling each pixel to a ...
1
vote
1answer
41 views

Is there any direct relation in between accuracy and loss while performing text classification using neural network?

I am trying to perform text classification using the deep recurrent neural network. My network is incurring a huge loss of 94%, 80% and sometimes 100% with certain accuracy. It is surprising that with ...
1
vote
0answers
33 views

How to learn a neural network based on a cost function in an unsupervised way?

I need to learn a neural network that predicts L (see equation below). So based on three points from a discrete trajectory I want to learn L by minimizing the equation below, which is a physical ...
2
votes
0answers
18 views

Pre-training without seeing data

Is there a solid reference on pre-training methods in deep neural networks which never see the actual inputs? Any such known thing in literature? I guess a more correct term is "initialization ...
2
votes
0answers
20 views

How does the Loss of a Neural Network effect training

Suppose I have a neural network, say an input layer taking in vectors $i \in \mathbb{R}^d$, a hidden relu layer, and then a softmax output layer with a cross entropy loss (with no biases added for ...
2
votes
0answers
33 views

Are Spiking Neural Networks The Next Big Thing? [closed]

Intel recently announced their Loihi chip as part of their "Neuromorphic Computing" research, which is optimized for spiking neural networks (SNNs). I found an example of a problem in which ...
1
vote
0answers
5 views

Interpreting the results of the paper of Kernalized correlation Filter KCF

Below is a snapshot of the output of the original paper of KCF.--> open link for full paper http://www.robots.ox.ac.uk/~joao/publications/henriques_eccv2012.pdf I have two questions: 1-in the ...
0
votes
0answers
19 views

1x1 convolution instead of Global Average Pooling

is it possible and useful to use a 1x1 convolution before the flatten and dense layer instead of the GAP? The 1x1 conv should theoretically select the most important Feature maps
0
votes
0answers
13 views

Evaluate result of a Detection algorithm based on the object's shape as Ground truth

I'm working on project where i have to detect small colored cars driving on rails from static camera (Bird's Eye-view see below image) The algorithm in short outputs first a mask image with only the ...
0
votes
0answers
10 views

Input Output Selection for multiple Models: Ensemble Stacking

I am trying to build an ensemble model. I may use wrong terminology in my question but in essence what my goal is, is to build multiple models who's output goes to a secondary model and I am ...
0
votes
0answers
7 views

Can we model a linearly increasing time series with an lstm without making series stationary

If I want to model an linearly increasing times series using LSTM, should I remove the trend and seasonality from the time series before feeding it to LSTM or can LSTM efficiently model non stationary ...
0
votes
0answers
9 views

How to know if the data size is not large enough for training deep learning models?

In general, in deep learning, how do we know if the size of training data is insufficient for the modeling task or how to know if adding more data will help approximate the function better and improve ...
0
votes
0answers
9 views

Calculating derivative of the activation function w.r.t input unit for Recurrent Neural Network(RNN)

Suppose we have the following structure of the RNN(many-to-one): where $\hat{y}(t)$ is a prediction at the end of the sequence, a(t) are hidden units and x(t) are input at each timestamp t. $s(t) = W^...
0
votes
0answers
23 views

SARIMAX - LSTM Evaluation of forecast results

I am trying to build a sales forecasting algorithm for more than 2000 products and now sticking around ARIMA/SARIMAX and LSTM models. As I can see, both models are okay to use the data without make ...
0
votes
0answers
20 views

Feature selection for neural network model

I am working with the Pamap2 sensor data for human activity recognition, using convolutional neural networks having these ...
0
votes
0answers
27 views

can we apply machine learning or Deep learning to this use case?

I am working on the recommendation system of collaborative filtering, where i have final dataframe with the four variables like user_id,product_id,reaction_id and rating.This gives a 3D sparse matrix. ...
0
votes
0answers
11 views

What does the loss function for a heteroscedasticity areatoric uncertainty mean in Bayesian Neural Network?

The following is the loss function to learn a Heteroscedastic uncertainty model: $ Loss = \frac{|| y - \hat{y} ||_2}{2 \sigma^2} + \frac{1}{2} \log \sigma^2 $ I am having trouble in understanding what ...
0
votes
0answers
9 views

Understanding adding up of gradients for branched variables in chain rule of calculus in the context of neural network backpropagation

I was trying to understand how we can calculate gradients in back propagation in the context of neural network from here. It says following: The forward expression involves the variables $x,y$ ...
0
votes
0answers
17 views

The best way to create a machine learning model/neural network that automatically searches for efficient available time slots in an employee's agenda

For an assignment I need to work out how to incorporate machine learning techniques/neural networks into automatic appointment planning. The algorithm needs to be able to be able to check all open ...
0
votes
0answers
13 views

What should be transfer learning model accuracy? [duplicate]

I have made base model for transfer learning and it is showing good accuracy, and even good confusion matrix is also showing good results Here is accuracy and losses for base model loss: 0.0566 - ...
0
votes
0answers
6 views

How many Test-videos are needed to evaluate a detection algorithm?

I've developed an object-detection algorithm based on combinations of available approaches in the literature (using classical approches). The algorithm is designed for a special application which ...
2
votes
2answers
43 views

heterogenous Neural Network output

My neural network (actually it is a CNN) must output the transformation matrix coefficient it should output tx and ty for translation that lay in the range of [0,200], 2 coefficients for shear [0.7, 1....
1
vote
0answers
10 views

Resnet model does not train with triplet loss, while VGG16 is able to train, why?

I am trying to do a transfer learning with ResNet50V2 model using triplet loss function. I have kept Include_top = False, input shape = (160,160,3) with Imagenet weights. The last 3 layers of my model ...
0
votes
0answers
19 views

Why don't we use the direct observation probability distribution in variational autoencoders?

In almost every code related to the VAE we generate x|z by simply getting it from the last layer of the decoder, e.g. using a sigmoid activation function. Why don't we sample from $\mathcal{N}(\mu, \...
2
votes
0answers
20 views

Avoiding exploding gradients when forget gate is 1 and input/output gates are 0

Initially, the cell state equation was $C_t = C_{t-1} + i_t \odot \tanh(w_xx_t + w_hh_{t-1})$. Then to avoid exploding gradients, we added a forget gate such that $$C_t = f_t \odot C_{t-1} + i_t \odot ...
0
votes
0answers
14 views

Why label encoding for categorical features should not be used for deep learning/neural network? [duplicate]

Many online posts(here and here) say label encoding would make NN/DL models think there is an ordering among categories, so it should not be used for categorical encoding for NN/DL models. But wouldn'...
0
votes
0answers
17 views

Why are point-wise nonlinearities equivariant to any permutation of the input and output indices of a network layer?

The statement from [1] says that: Pointwise nonlinearities such as ReLU and sigmoid are already equivariant to any permutation of the input and output indices (of a network layer), which includes ...
6
votes
1answer
78 views

What is the application for using the `Boltzmann Machines`?

I've came across this post from wikipedia about Boltzmann machines, which concludes that it is not a model that is usually used in practice, and that its' ...
0
votes
0answers
8 views

ArcFace and Image Embedding

I am learning the ArcFace paper since I saw most winning image classification/retrieval solutions used it. I used it on MNIST, CIFAR10, CIFAR100 and I noticed the accuracy was very low compared to ...
1
vote
0answers
21 views

Best loss functions and data scaling methods for dataset with large variance in output variables? [closed]

I work on a regression model using feed forward neural networks. I have a dataset of $\sim 10^7$ entries, connecting a vector 𝑋 with a vector 𝑌. The dataset is such that the elements of $Y$ have a ...
0
votes
1answer
15 views

What is multi-scale architecture?

When I was reading the paper "Density Estimation using Real NVP", I have found the term "multi scale architecture". We implement a multi-scale architecture using a squeezing ...
0
votes
0answers
13 views

What exactly are nodes or units in neural networks?

Let's say I have a simple feed-forward network with 1 hidden layer of 500 units/nodes with sigmoid activation function for binary classification of "customer churn" or "customer don't ...
2
votes
2answers
136 views

Optimization for the autoRec algorithm

In its original paper, autoRec proposes the following algorithm: $min_{\theta} \sum_{i = 1}^{n} \|r^{i} - h(r^{i}, \theta)\|^2_{\mathcal{O}} + \frac{\lambda}{2} (\|W\|_{F}^2 + \|V\|_{F}^{2})$ where $\|...
2
votes
1answer
65 views

The gradient of neural networks w.r.t one-hot encoded inputs

One-hot encoding as raw inputs for deep learning models can find its applications in many domains, such as bioinformatics, NLP, chemistry and so on. Suppose we trained a neural network $f(x)$ with $x$ ...
0
votes
0answers
19 views

Decision Boundary formula + Visualization

So i'm trying my best to try and understand the formula for a decision boundary regarding a binary classification problem using an Artificial Neural Network. I'm using Tensorflows Playground for this. ...
0
votes
0answers
12 views

Mathematics behind Activation functions [duplicate]

In Machine learning we use activation functions to give non-linearity to the output of neuron. But what is the exact non-linearity in this context? How it differs in different activation functions(e.g....
0
votes
1answer
19 views

Resize image in object detection task of computer vision

In object detection, they usually resize by keeping the ratio the same as the original image, which usually names "letterbox" resize. My question is: Why we need to do that? As I see with ...
0
votes
1answer
54 views

Overfitting in Machine Learning and mathematical background of it

Simply we know that when we train a model if validation error stops improving i.e. if it started to increase we called it "overfitting". If there is a high variance with low bias we can ...
1
vote
1answer
22 views

Can large # of epochs or smaller batchsize compensate for smaller data size in training lstms

I have about 40 time series (40 products) of weekly sales for 3 years ( = 156 data points for each series). So, in total I have about 6240 data points. To train a stateful or stateless lstm for ...
2
votes
1answer
22 views

Representing bias term in simple linear neural network(linear regression) using analztical solution

Assume that output $y$ depends on input $x$ and some noise $\epsilon \sim N(0,\sigma^2)$. $$y = f(x) + \epsilon$$ Suppose we want to model relationship mentioned above using linear neural network: $$ \...
1
vote
1answer
34 views

Optimization of Linear Autoencoder with SGD

I'm interested in the Linear Autoencoder(LAE), and I knew that, at convergence point, the subspace LAE learns is the same as the subspace PCA learns up to linear transformations. Also, the loss ...