First of all, you should distinguish between keypoints (feature points) and their descriptors (feature vectors constructed on the base of an image patch taken around corresponding keypoints). In theory, optimal detection and description of keypoints should be deeply connected. However, in practice, one can detect keypoints and construct their descriptors using almost unrelated methods.
Keypoints are the most unique (or informative) points and most stably detectable points on images. Of course, informativeness of points depends on image type under consideration. For ordinary images, informative and stable points lie on borders of uniform regions (since our visual world approximately consists of a set of smooth surfaces). Consequently, most detectors of keypoints are based on maxima of partial derivatives (gradient, Laplace operator, etc.). Of course, this is a simplification, so there are many different implementations of these detectors, which have somewhat different performance (and speed) on different images. If you don’t want (or cannot) to select more suitable detector, the simplest way is to take any general detector, e.g. on the base of difference of Gaussians.
If your images are very specific (very unnatural), the only way is to automatically construct a good detector using the criterion of its informativeness. Different approaches to this problem exist originating from statistics and information theory. Or one can try adopting something like convolution autoencoders with pooling.
The same is true for descriptors, but their influence on performance more significant, and methods for constructing features are even more diverse. However, considerable part of hand-crafted feature transforms is very similar to features that can be obtained automatically by PCA, ICA, autoencoders, etc. Hand-crafted features can be calculated faster (due to code optimization), and don’t require off-line learning. But if your images are of really unknown nature, then it is better to learn feature transforms from some training set of images. However, there is a questing concerning their invariance. Most methods with hand-crafted feature transforms perform affine (or rotation and scale) normalization of image patches before calculating features in order to achieve invariance to spatial transformations. Right now, no methods exist, which can learn arbitrary invariants.
Indeed, in bags of visual words spatial arrangement of keypoints is ignored. This makes comparison of images very fast. And, indeed, we lose valuable information this way, and in some cases remain information could not be enough for good recognitions. However, an alternative solution is to perform keypoint matching, which is much slower. Well, there are also intermediate solutions, in which some spatial information is incorporated, but full point-to-point matching is not needed.
