# How to calculate disparity of two images in matlab?

I have two images and I am trying to calculate the disparity between them using sum of squared distances and reconstruct disparity in 3D space. Do I need to rectify the image before calculating disparity?

The following are the steps that I have done so far for disparity map computation(I have tried with rectification and without rectification but both are returning all zeroes disparity matrix).

For each pixel in the left image X,
Take the pixels in the same row in the right image.
Separate the row in right image to windows.
For each window,
Calculate the disparity for each pixel in that window with X
Select the pixel in the window which gives minimum SSD with X
Find the pixel with minimum disparity among all windows as the best match to X


Am I doing it correctly?

How can I visualise the 3D reconstruction of the disparity as scatter plot in matlab?

First off, I think you need to clarify what it is you mean by "disparity". The sum of squared distances is an easy measure, however it is fraught with problems when you consider practical factors.

To the human eye, two pictures may look similar and yet have vastly different sum of squared errors. The sum of squared distance measure is extremely sensitive to even the slightest difference between pixels.

A great paper that reconciles practical considerations and suggests other measures is this one http://www.mdpi.com/2072-4292/2/3/794 by Dowd and Fennel in Remote Sensing. It is geared towards oceanography/atmospheric sciences, but naturally it has applications to other fields as well.

Now for the algorithm that you're looking for: What you wrote doesn't seem quite clear to me, so here's the pseudocode for what I mean.

Let $(a,b)$ be the co-ordinate of a pixel in image $A$ with RGB value $(x,y,z)$ and $(a,b)$ be the co-ordinate of a pixel in image $B$ with RGB value $(m,n,p)$. The distance for pixel $(a,b)$ is therefore $(x-m,y-n,z-p)$, and you do a loop to go through all the co-ordinates. To compare image $A$ to image $B$ you have to compare the RGB values of the same co-ordinate. So you pictures have to be of the same size, pixel-wise.

If your two images are exactly the same, then the disparity would be 0 for every pixel. You either have to use two separate cameras to take the images, or take them with a single camera from two different locations. The best way to do 3D reconstruction is to use a calibrated stereo pair of cameras. Here is an example of how to do that using the Computer Vision System Toolbox for MATLAB.

You'll probably need to correct for gamma, since in a gamma encoded image the pixel values are not the brightness, but a function of brightness.