Questions tagged [neural-networks]
Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.
18 questions from the last 30 days
1
vote
0
answers
19
views
Preprocessing and model selection strategies
I am working on a fault detection problem where each sample is a time series labeled with a specific type of fault. I am using a CNN model and a validation set for hyperparameter tuning. Currently, I ...
- 77
1
vote
1
answer
11
views
Varying sequence lengths between classes in LSTM
I am working on a project where the goal is to predict whether students in an online course will drop out of the course. The course is divided into 20 course weeks. For each week, I have certain kinds ...
- 11
1
vote
0
answers
17
views
Can the closed-form solution for ridge regression be used in training neural networks?
Is it established that the closed-form solution of ridge regression can be used during the training of neural networks? If so:
What are the potential benefits of using it?
In what scenarios would ...
- 31
2
votes
0
answers
28
views
Does the ill-conditioning of the design matrix affect the ill-conditioning of the Hessian in the context of DL?
I know that when we use the square loss as our cost function in DL, the ill-conditioning of the Hessian is directly tied to that of the design matrix. Does this apply to other cost functions ?
If so, ...
- 31
0
votes
0
answers
10
views
What is the best way to set weights for weighted MSE in multi output regression? [duplicate]
I am working on a regression task where the goal is to predict 6 scalar output values from a given input. The input consists of decaying signal data, and the outputs are the parameters of the signal ...
- 11
1
vote
2
answers
34
views
How does a single layer/single unit with Adam optimizer network work?
I'm very new to ML and I'm trying to mess around with Linear Regression. I tested sklearn's LinearRegression model and then wanted to compare the results to a very simple neural network.
I created a ...
- 111
3
votes
0
answers
19
views
Reward and Penalty Design in reinforcement learning
I hope you're all doing well.
I am currently working on a reinforcement learning problem to solve an optimization problem in wireless networks and I'm having troubles with designing the reward and ...
1
vote
1
answer
20
views
Learnability of boolean formulae by neural networks using back propagation?
I've been researching neural networks and boolean formulae. From my efforts, it doesn't seem that neural networks can generally learn boolean formulae using back propagation. This makes sense ...
- 143
0
votes
0
answers
16
views
"Inflating" learning rates in diminishing gradient areas for NN training
In neural net training, nowadays tanh and sigmoid activation functions in hidden layers are avoided as they tend to "saturate" easily. Meaning, if the x value plugged into tanh/sigmoid is ...
- 101
2
votes
2
answers
37
views
Predicting the probability distribution of a deterministic dataset
In classical machine learning regression, we often assume the target variable $y$, given an input $x$, follows a probability distribution, allowing us to model and predict not just the expected value ...
- 121
0
votes
0
answers
16
views
Why do VAEs work?
I am currently reading into Variational Autoencoders, and although I kind of understand the mathematical background described in the original paper (Auto-encoding Variational Bayes), I am struggling ...
0
votes
0
answers
16
views
Geometric structure of the output of a random weight neural network fed with random data
Take a model with random weigths:
...
0
votes
0
answers
15
views
Embeddings in time series prediction
Increasingly, I’ve noted that embeddings are used in pure prediction ML tasks. For example, instead of predicting whether user i will purchase item i and thereby adding thousands or millions of inputs ...
- 3,520
3
votes
1
answer
163
views
Questions on backpropagation in a neural net
I understand how to symbolically apply back propagation, calculate the formulas with pen and paper. When it comes to actually using these derivations on data, I have 2 questions:
Suppose certain ...
- 599
1
vote
1
answer
27
views
Reason for softmax approximation in Ian Goodfellow's deep learning book
In section 6.2.2.2 (equation 6.31) they state:
Overall, unregularized maximum likelihood will drive the model to learn parameters that drive the softmax to predict the fraction of counts of each ...
- 11
0
votes
0
answers
14
views
Why does my test loss and test evaluation metrics fluctuate?
I am fine-tuning the resnet18 model with additional classifiers. What I observed during the training process, is that test loss and other test evaluation metrics (AP, AUC) seem to fluctuate as you can ...
- 1
0
votes
1
answer
28
views
GNNs with higher order adjacency matrices
Usually, the adjacency matrix stores information about direct connections of nodes in a graph.
The information from k-th neighbours is passed-on at k-th layers of GNNs, as described in the original ...
0
votes
0
answers
17
views
Why does the image classification model perform worse when augmenting only minority class
I have a problem of data imbalance (1:10 ratio) for image classification tasks.
To cope with it, I tried different imbalance training strategies, including weighted loss function, different loss ...
- 1