# In neural networks, how to compute the mean square error (MSE) in gradient update when using a minibatch?

I've been using a siamese neural network for the binary classification of biological data. I've implemented a Torch version of this algorithm, including a stochastic gradient update function.

At each iteration, this function reads one input profile and its corresponding target label (true/false), applies the back-propagation technique and finally generates one predicted value, that I will use in computing the confusion matrix. This means, I have 1 input profile, 1 target, and 1 output predicted value.

To check the performance of this gradient update function, I can compute the mean square error (MSE = (targetValue - predictedValue)^2 ). This is very useful.

Then I wanted to implement a mini-batch gradient update function. This is a function that reads N input profiles, and their corresponding N target labels (true/false). But, since my siamese neural network architecture has only 1 singular final output, it produces 1 singular output predicted value.

My problem is that, in this case, I cannot (or I don't know) how to compute the mean square error (MSE). I could do it if I had N output values, but since I only have 1 predicted value, what should I do?

Do you guys have any suggestions? How to compute the MSE error in minibatch gradient update?

Or am I doing something wrong?

My Torch code:

Gradient update for the siamese neural network:

function gradientUpdate(generalPerceptron, input_profile, targetValue, learningRate);

function input_profile:size() return #input_profile end
local predictionValue = generalPerceptron:forward(input_profile);

if predictionValue*targetValue < 1 then
generalPerceptron:updateParameters(learningRate);
end

local meanSquareError = math.pow(targetValue - predictionValue,2);

return generalPerceptron;
end


Minibatch gradient update for the siamese neural network:

function gradientUpdateMinibatch(generalPerceptron, input_vector, targetVector, learningRate)

function input_vector:size() return #input_vector end
local predictionValue = generalPerceptron:forward(input_vector);

local target_array_tensors = -targetVector;
generalPerceptron:updateParameters(learningRate);

return generalPerceptron;
end


If you want N output values instead of 1, you should implement generalPerceptron:forward() such that if it receives an NxM input matrix (i.e. N samples with M features), it outputs N values. I.e. it should perform a matrix multiplication between the input and the weights of the network.

EDIT: based on your comment. If you cannot modify forward(), you can just iterate over the input samples one at a time. I.e. you take one input sample, get a prediction using forward() on that input sample only, and use that to calculate the gradient. If you want to do minibatch with MSE, you would do something like this (pseudocode-ish):

For each minibatch:
For each x_i in this minibatch:
yhat = generalPerceptron:forward(x_i)
error = 0.5*(target - yhat)^2 -- squared error

• I think I cannot modify the forward() function, but I'm not sure... Apr 23, 2016 at 17:37