Before I begin my answer and because you use both terms in your question, I'd like to note that:
$$
loss \; function = cost \ function
$$
The cost function is the way your model evaluates its performance. The lower the cost, the closer your predictions are to the targets and thus the better your model is performing. This is the definition of the loss function. So, the goal of the whole learning process is to simply lower the value of the cost function.
How do you proceed to minimize the cost? SGD! Once you reach a local minima and the cost is at its lowest value, you can say that that's the closest your predictions can get to the targets.
In case something isn't exactly clear I'll answer your questions one by one:
When calculating the derivative of the the cost with respect to the weights, why is it that we move in the direction of minimising this cost?
Because that's the goal of the whole training procedure: to minimize the cost. The cost shows you how close your predictions are to the targets.
Why does this optimise the model and cause weight adjustment that produces better predictions. Is the local minima directly linked to the loss function?
Yes the local minimum is referring to the loss function. I.e. it's a place in the function where, whichever direction you choose will increase its value.
Is it intuitive that the loss function creates error metric 'x' and that finding the local maximum through gradient descent for example would simply increase the error and thus weight adjustment moves in the wrong direction?
No, the (training) error is measured by the loss function. By minimizing the loss function, by definition, you are reducing the error.
