Is it possible that grid search would fail in two dimensional feature space? Grid search suffers from the curse of dimensionality. But is there any case(any hypothetical distribution of data) in a two dimensional feature space where the data’s binary classification using Grid Search would yield very poor results? 
 A: Grid search at appropriate resolution will always produce excellent results. The problem arises when getting to that appropriate resolution is intractable. Numerical approximations become infeasible very quickly as the dimension of the data increases, but it's entirely possible to have an intractably large space to explore in just two dimensions. Consider a function sampled from Brownian Motion: there will be lots of local optima, the data is not smoothed by increasing resolution, and the spread of the data relative to the global mean doesn't ever really decrease over the entire domain (as opposed to something like the normal pdf where the values are all approximately equal to the mean (0) for the entire support, except within the 99% confidence interval). 
So to summarize, think of it this way: pick some minimum resolution R for your search. In high dimensional space, you will consider grid search intractable if you need at minimum K iterations to search your space at resolution R. If in two dimensions you need more than K iterations to search at R resolution, you have exactly the same problem as you had in the higher dimension search.
