Inconsistency in unit on gradient descent equation The update algorithm for gradient descent is 
$$\theta_j = \theta_j - \alpha \frac{\partial}{\partial \theta_j}J(\theta)$$
$ \frac{\partial}{\partial \theta_j}J(\theta)$ have the unit of cost per unit $\theta$ but $\theta_j$ have the unit of $\theta$ and constant $\alpha$ is dimentionless, why is it valid that we can use subtraction for two element with different units? 
 A: As was concluded in the discussion in comments, dimensional analysis would necessitate that the relevant component of $\alpha$ is in fact in the units necessary to make 
$$\alpha_j \frac{\partial}{\partial \theta_j}J(\theta)$$
have the same units as $\theta_j$
A: For the same reason that the slope of a line is not the "run", but is instead "rise" over "run", a gradient isn't a displacement in your theta-parameter space... anyone telling you otherwise is wrong. This is why the units don't match as you noted. However, the fundamental property of a gradient is that the directional derivative of a function is maximized in a direction PARALLEL to the gradient. It makes sense because gradient vector components are all the "rise" of the function divided by the "run" in each parameter (i.e. they're the slopes in each direction). If the slope of a tangent to your function (hyper-)surface is 5 times stronger in one parameter than another, you will want to move 5 times more in the strong parameter than the other. If the units are bothering you, remember that you can multiply (or divide) the gradient vector by any unit and by any scalar (like minus alpha) and you will still be parallel (or anti-parallel) to it.
