I have data like Person $A$ like movies
['X','Y', 'Z'] and he dislikes
['V']. Person $B$ like movies
['X','L','V'] and dislikes
['Y']. like wise so many users. What could be a good algorithm to find mean difference of users' tastes?
What you want to do is called "Collaborative Filtering". Searching the web will offer you a tremendous amount of resources for this topic, but I truly recommend this paper:
In section 3. Memory-Based Collaborative Filtering Techniques you'll find the basic techniques to find users with similar taste. They generally consist of selecting a metric for a nearest-neighor-approach plus some modifications on the user-item-matrix (how to deal with missing values / how to treat items which have not been rated this often etc.).This is a good point to start.
After grasping the basic ideas you may want to try out some sophisticated techniques like Singular Value Decomposition, which has been successfully applied in the Netflix-Price (you'll find a link to this and other techniques in section 4. Model-Based Collaborative Filtering Techniques of the recommended paper).
If you have some bucks to spend, I also recommend "Programming Collective Intelligence" by Toby Segaran, which approaches this topic in a very very practical way.
If you represent each movie as a categorical variable with 3 levels (like, unspecified, dislike) you can do any type of clustering analysis on your users with these covariates.