I have a training dataset of images: X
(Visual) and Y
(Infrared). Each set has $300$ training examples. I extract feature vectors from both sets of images. Thus my X
and Y
datasets are respectively $300\times 1920$ and $300\times 1536$ where $300$ is the number of sample observations (in this case images) and $1920$ & $1536$ are respectively the length of the feature vectors in visual and infrared spectrum.
A testing dataset consists of images of different subjects in both visual spectrum and infrared spectrum. I will have the visual spectrum data as my gallery. I will have the infrared images as the probe data. For each probe image, I need to retrieve the corresponding visual image from the gallery by using some kind of similarity measure.
Basic algorithm idea:
- Start Training Phase. Read the images from visual and infrared spectrum dataset.
- Get the feature vectors by using desired descriptors and populate the matrices X and Y.
- Use CCA for subspace learning. Get the projection matrices
Wx
andWy
. - Start Testing Phase. Given the gallery images (visual), read these images and use the
Wx
transformation matrix to convert them into the CCA subspace. - For each probe image, convert it into the CCA subspace by using the
Wy
transformation and compare with each images of the gallery and compute a matching score. - The image (or its label) in the gallery having the maximum score is returned.
Could anyone tell me if my approach is correct or not? Pointing me in the right direction would also be helpful. Please see the following paper for reference: Yi et al. 2007, Face Matching Between Near Infrared and Visible Light Images.
I work in Matlab and use the following command to perform CCA:
[Wx,Wx,r,U,V] = canoncorr(X,Y); %// DO CCA
The output I get is this :
Name Size Bytes Class Attributes
Wx 1920x297 890880 double
Wx 1536x297 712704 double
U 300x297 27840 double
V 300x297 27840 double
r 1x297 464 double
As was explained to me on StackOverflow:
The projection matrices are
Wx
andWy
since they transformX
andY
into the new space.The resulting projections of
X
andY
into the new space areU
andV
, respectively.The
r
vector represents the entries of the correlation matrix betweenU
andV
, which is a diagonal matrix.