User Tom Hale

3

11

votes

4

answers

6k

views

Convolution with a non-square kernel

Dec 4 at 4:38 Taro Kiritani 141

37

47

votes

5

answers

37k

views

Why is tanh almost always better than sigmoid as an activation function?

machine-learning neural-networks backpropagation sigmoid-curve

Aug 21 at 0:42 Osama El-Ghonimy 21

10

votes

2

answers

453

views

Etymology of multi-armed bandit

reinforcement-learning multiarmed-bandit

May 19 at 11:45 Tim♦ 88.4k

3

10

votes

2

answers

8k

views

Difference between pooling and subsampling

neural-networks conv-neural-network computer-vision

May 8 at 10:29 KeepItSimple 1

3

10

votes

4

answers

4k

views

Regularisation: why multiply by 1/2m?

regularization

Nov 24 '19 at 23:37 Keyan P 101

2

votes

1

answer

562

views

Is a GAN's discriminator loss expected to be twice the generator's?

machine-learning neural-networks loss-functions gan

Apr 13 '19 at 8:43 shimao 17.1k

5

votes

0

answers

409

views

Usage of dropout in convolutional GANs with batch norm?

neural-networks gan dropout batch-normalization

Apr 9 '19 at 6:51 Tom Hale 1,791

6

votes

2

answers

1k

views

Etymology of “Adam” algorithm for gradient descent

terminology gradient-descent history adam

Sep 21 '18 at 11:55 amoeba 85.3k

1

2

votes

0

answers

1k

views

Skip connections: Benefits of skipping extra layers

machine-learning neural-networks conv-neural-network

Aug 4 '18 at 9:12 Tom Hale 1,791

1

6

votes

1

answer

2k

views

Transfer learning: How and why retrain only final layers of a network?

machine-learning neural-networks backpropagation transfer-learning

May 31 '18 at 9:36 user20160 25.5k

1

vote

1

answer

124

views

Notation to represent the batch-normalised value of x

machine-learning neural-networks normalization notation batch-normalization

May 11 '18 at 11:34 Peter Flom 91k

0

votes

1

answer

152

views

Error in converting activation function from ReLU to ELU

neural-networks python

Mar 16 '18 at 11:40 Jim 1,684

5

28

votes

1

answer

32k

views

What is the derivative of the ReLU activation function?

self-study neural-networks

Mar 15 '18 at 17:28 Jim 1,684

5

votes

2

answers

659

views

Why is two-sided gradient checking more accurate? [closed]

optimization gradient-descent gradient numerics

Feb 26 '18 at 16:21 Sycorax♦ 64.7k

2

votes

1

answer

2k

views

Working for Logistic regression partial derivatives

logistic backpropagation derivative gradient

Feb 24 '18 at 13:32 Tom Hale 1,791

1

vote

0

answers

92

views

Intuitive understanding of Extreme Learning Machine [duplicate]

machine-learning neural-networks references intuition

Feb 2 '18 at 10:09 kjetil b halvorsen♦ 51.9k

1

0

votes

1

answer

1k

views

Should one-hot output encoding be used in backpropagation?

neural-networks backpropagation categorical-encoding

Dec 22 '17 at 20:20 user0 4,577

3

votes

3

answers

1k

views

Backward propagation without a loop over training examples

machine-learning neural-networks matlab backpropagation

Sep 20 '17 at 9:45 asterio gonzalez 121

5

8

votes

1

answer

785

views

Why is sqrt(6) used to calculate epsilon for random initialisation of neural networks?

machine-learning neural-networks random-generation

Jul 23 '17 at 14:13 seanv507 3,754

0

votes

1

answer

67

views

Simplification of case-based logistic regression cost function

logistic

Jun 29 '17 at 8:58 Tom Hale 1,791

Stack Exchange Network

current community

your communities

more stack exchange communities

Next privilege

Next badge

20 Questions

Convolution with a non-square kernel

Why is tanh almost always better than sigmoid as an activation function?

Etymology of multi-armed bandit

Difference between pooling and subsampling

Regularisation: why multiply by 1/2m?

Is a GAN's discriminator loss expected to be twice the generator's?

Usage of dropout in convolutional GANs with batch norm?

Etymology of “Adam” algorithm for gradient descent

Skip connections: Benefits of skipping extra layers

Transfer learning: How and why retrain only final layers of a network?

Notation to represent the batch-normalised value of x

Error in converting activation function from ReLU to ELU

What is the derivative of the ReLU activation function?

Why is two-sided gradient checking more accurate? [closed]

Working for Logistic regression partial derivatives

Intuitive understanding of Extreme Learning Machine [duplicate]

Should one-hot output encoding be used in backpropagation?

Backward propagation without a loop over training examples

Why is sqrt(6) used to calculate epsilon for random initialisation of neural networks?

Simplification of case-based logistic regression cost function