The Stack Overflow podcast is back! Listen to an interview with our new CEO.
130

These are not very strict terms and they are highly related. However: Loss function is usually a function defined on a data point, prediction and label, and measures the penalty. For example: square loss $l(f(x_i|\theta),y_i) = \left (f(x_i|\theta)-y_i \right )^2$, used in linear regression hinge loss $l(f(x_i|\theta), y_i) = \max(0, 1-f(x_i|\theta)y_i)$, ...


93

First of all, there's no reason that an agent has to do the greedy action; Agents can explore or they can follow options. This is not what separates on-policy from off-policy learning. The reason that Q-learning is off-policy is that it updates its Q-values using the Q-value of the next state $s'$ and the greedy action $a'$. In other words, it estimates ...


14

The only difference is that : The sigmoid neuron is able to output values between 0 and 1. The stochastic binary neurons only can output either a 0 or a 1 with the probability given by the sigmoid activation function.


13

According to Wikipedia, there are 4 main types of artificial neural network learning algorithms: supervised, unsupervised, reinforcement and deep learning. Unsupervised learning algorithms: Perceptron, Self-organizing map, Radial basis function network Supervised learning algorithms: Backpropagation, Autoencoders, Hopfield networks, Boltzmann machines, ...


13

As you mentioned, most neural networks are based on general simple abstractions of the brain. Not only are they lacking in mimicking characteristics like plasticity, but they do not take into account signals and timing as real neurons do. There's a fairly recent interview, that I felt was appropriate for your specific question, Machine-Learning Maestro ...


12

On-policy methods estimate the value of a policy while using it for control. In off-policy methods, the policy used to generate behaviour, called the behaviour policy, may be unrelated to the policy that is evaluated and improved, called the estimation policy. An advantage of this seperation is that the estimation policy may be deterministic (e.g. greedy)...


12

I think there's some confusion here. The reason you have vanishing gradients in neural networks (with say, softmax) is wholly different from RNNs. With neural networks, you get vanishing gradients because most initial conditions make your outputs end up on either the far left or far right of your softmax layer, giving it a vanishingly small gradient. In ...


11

I think this is more about Engineering Problem Solving. Most successful engineering projects do not duplicate expert reasoning or or the expert's nature exactly. They solved the problem in a different way. For example washing machines use a different technique than humans, airplanes use different dynamics than birds. If you are duplicating Expert ...


11

Not much --- arguably nothing --- has so far been learnt about brain functioning from artificial neural networks. [Clarification: I wrote this answer thinking about neural networks used in machine learning; @MattKrause (+1) is right that neural network models of some biological neural phenomena might have been helpful in many cases.] However, this is perhaps ...


8

It is certainly not true that the human brain only uses "a few" convolutional layers. About 1/3 of the primate brain is somehow involved in processing visual information. This diagram, from Felleman and Van Essen is a rough outline of how visual information flows through the monkey brain, beginning in the eyes (RGC at the bottom) and ending up in the ...


8

According to Prof. Andrew Ng (see slides on page 11), Function h(X) represents your hypothesis. For fixed fitting parameters theta, it is a function of features X. I'd say this can also be called the Objective Function. The Cost function J is a function of the fitting parameters theta. J = J(theta). According to the Hastie et al.'s textbook "Elements of ...


8

Make the identity of the agent one of the features, and train on all data. Probably train on a mini-batch of eg 128 agents at at time: run through the time-series from start to finish for those 128 agents, then select a new mini-batch of agents. For each mini-batch, run a slice of say 50 timesteps, then backprop. Keep the end states from that slice, and run ...


7

The difference between Off-policy and On-policy methods is that with the first you do not need to follow any specific policy, your agent could even behave randomly and despite this, off-policy methods can still find the optimal policy. On the other hand on-policy methods are dependent on the policy used. In the case of Q-Learning, which is off-policy, it ...


7

First of all, what actually policy (denoted by $\pi$) means? Policy specifies an action $a$, that is taken in a state $s$ (or more precisely, $\pi$ is a probability, that an action $a$ is taken in a state $s$). Second, what types of learning do we have? 1. Evaluate $Q(s,a)$ function: predict sum of future discounted rewards, where $a$ is an action and $s$ ...


7

My father went to university shortly after World War II and he used to tell me about how he once did a long Fourier transformation and how they used those very long paper rolls, as they did it in a paper pencil approach. When I went to school in the 80s, people used to think, that in some time in the future everybody would have to deal with computers. And ...


6

Computational learning, more concretely the probably approximately correct (PAC) framework, answers questions like: how many training examples are needed for a learner to learn with high probability a good hypothesis? how much computational effort do I need to learn with high probability such hypothesis? It does not deal with the concrete classifier you are ...


5

That can actually sound a little strange within community of statisticians, but I am pretty sure that most of machine learning algorithms can be formulated as a functional minimization problems. That means that this is going to be covered with mathematical optimization. The other thing is that you will probably need calculus and linear algebra to understand ...


5

That depends on the image. In general the answer is 100%, but in different strengths, depending on both the image and the specific network. In fact, encouraging full-network usage is one of the objectives of the "dropout" technique. This has been written about extensively in many places, but this blog post is a nice quick introduction. I'm not aware of in-...


5

The whole point of that initialisation scheme (sometimes called "Xavier initialisation") is to mitigate the problem of disappearing gradients caused by the form of many activation functions. Consider the common sigmoidal activation function $s(z) = \frac{1}{1+e^{-z}} = \frac{e^z}{e^z + 1}$. Its gradient $\sigma'(z) = s(z)[1-s(z)]$ approaches $0$ away from ...


4

This field is evolving rapidly. Just a few days ago, the results of the imagenet 2014 challenge have been published. It will take some time until all the papers are available. If you want to solve these kinds of problems, the take away message is that most, if not all classical solutions to the problem are obsolete. The way to go (and it probably won't ...


4

Since "good" is a relative term, "good results" is a very ambiguous criterion. In addition, as @Nick Cox mentioned in his comment, it's impossible to cover the entire AI field, even if "good results" criterion would have a clear interpretation. Therefore, I think that, instead of a summary, it makes sense to present the state of AI research (in terms of ...


4

The recommended thing to do when using ReLUs is to clip the gradient, if the norm is above a certain threshold, during the SGD update (suggested by Mikolov, see http://arxiv.org/pdf/1211.5063.pdf) This requires another hyperparameter, the threshold. The suggestion from the referenced paper is to sample some gradients to get an idea of the (non-exploding) ...


4

The idea is to learn the bias weights but have the activation fixed at 1. Anything else would make it an additional ordinary unit.


4

Exploration might be an issue. Are you sure the algorithm tries all legal actions during the training? Setting a very high initial estimate for all the Q-values will encourage exploration at the start of the training. You could also try "soft selection" where you randomly select an action other than the one with highest Q-value some of the time.


4

I think that using not fully trained agents for difficulty below grandmaster might be hard to do, because the designer of training process has to focus both on creating the "best" agent in the end AND setting the right cutoffs in between. Given the often stochastic nature of balancing exploration and exploitation during the learning (e.g. by using $\epsilon$-...


4

The Spark documentation for this kind of thing doesn't seem very thorough, so I looked at the source. There's a comment here saying // Need not divide with the norm of the given vector since it is constant. This seems consistent with the following code. So, it seems that findSynonyms doesn't actually return cosine distances, but rather cosine distances ...


4

In Andrew NG's words- "Finally, the loss function was defined with respect to a single training example. It measures how well you're doing on a single training example. I'm now going to define something called the cost function, which measures how well you're doing an entire training set. So the cost function J which is applied to your parameters ...


4

Bagging in general is an acronym like work that is a portmanteau of Bootstrap and aggregation. In general if you take a bunch of bootstrapped samples of your original dataset, fit models $M_1, M_2, \dots, M_b$ and then average all $b$ model predictions this is bootstrap aggregation i.e. Bagging. This is done as a step within the Random forest model algorithm....


4

The behavior that you describe is called concept drift. Mostly concept drift is due to a natural change in the underlying data. In your case, you are in a game against the fraudsters, so you might want to be even more strict. In order to know whether your model still performs well, due to a change in the fraud patterns or a natural change, most methods ...


4

A convolutional layer with 1x1xC input shape and 1x1xC' output shape is equivalent to a fully connected layer. The architectures are the same. A "softmax layer" typically means a linear layer followed by softmax activation. There is no additional hidden layer.


Only top voted, non community-wiki answers of a minimum length are eligible