Suppose we have some training set $(x_{i}, y_{i})$ for $i = 1, \dots, m$. Also suppose we run some type of supervised learning algorithm on the training set. Hypotheses are represented as $h_{\theta}(x) = \theta_0+\theta_{1}x_{1} + \cdots +\theta_{m}x_{m}$. We need to find the parameters $\mathbf{\theta}$ that minimize the "distance" between $y_i$ and $h_{\theta}(x)$. Let $$J(\theta) = \frac{1}{2} \sum_{i=1}^{m} (y_{i}-h_{\theta}(x_i))^{2}$$

Then we want to find $\theta$ that minimizes $J(\theta)$. In gradient descent we initialize each parameter and perform the following update: $$\theta_j := \theta_j-\alpha \frac{\partial}{\partial \theta_{j}} J(\theta)$$

>**Question.** What is the key difference between batch gradient descent and stochastic gradient descent? 

Both use the above update rule. But is one better than the other?