Understanding how good a prediction is, in logistic regression I have fit a logistic regression model with four features and now I am able to use it for prediction.
When I plot the data points from my learning sample (i.e. I select two features, plot them in two axis), I see that there is more density of points in an area than in other areas. What I would like my model to return is not only the probabilities, but also a number that tells me how good these probabilities are (e.g. a p-value). I imagine that depending on the region the prediction is made, and the density of the sample points in that concrete region, the better the prediction, and conversely.
Is there a way of obtaining a value that tells me how good my prediction is? 
In concrete, I am using python scikit logistic regression package, although the answer can be more broad.
 A: The key thing to check first is the model's calibration, either using the bootstrap to correct for overfitting or using a huge independent sample not used for model development or fitting.  The best way to assess calibration is using a loess smooth nonparametric regression.  Once you establish calibration you can go on to predictive discrimination using the pseudo $R^2$ and Somers' $D_{xy}$ rank correlation coefficient, or a simple translation of it to the $c$-index AKA concordance probability or AUROC.  The Brier score is an excellent addition to all this.
A: I may be wrong with exactly what it is that you are looking for, but if you are worried about the overall distribution of your prediction quality, then I would borrow from machine learning validation tools since this is exactly what they are interested in as well. 
Here, you could do a 10-fold cross-validation (or a single hold out sample but you couldn't get information about the distribution of your metric over multiple samples of your data) of an area under the ROC curve metric (AUROC), for instance, if you would like to see how your predictions behave with different class cut-off probabilities. 
Another metric could be the mean squared error if you prefer that loss function.
A: You can return r squared,  p value and chi squared for a logistic regression.   Try using stats models,instead of Sci kit.   Stats models returns all of this data automatically 
