Your second question has been answered; as such I will attempt to answer your first question "Is there anything else that I can do?":
As time is the commodity of most value, I would recommend spending time on identifying the cause rather than randomly changing things on gut feel (precision and recall do not always answer question on bias and variance nor the appropriateness of the model chosen for classification. As such:
In your neural network implementation determine if you have a high bias or variance (e.g., see here), i.e. is your high precision and low recall due to under fitting High bias or over fitting High variance your positive examples as the methods for solving these issues differ from those for high variance, i.e.:
- Getting more training examples -Fixes high variance
- Trying smaller sets of features -Fixes high variance
- Increasing lambda -Fixes high variance
- Adding features -Fixes high bias
- Adding polynomial features -Fixes high bias
- Decreasing lambda -Fixes high bias
Examining the cases that are incorrectly classified can help determine why the model failed to classify these correctly (graphing and colouring the features on 2d plots can help sometimes (feature scaling / compression may be of use/ can make this possible in some cases).
Other good options include using a different architecture on your neural network, a different algorithm or modified features per below:
Consider treating your problem as anomaly detection, i.e. When normalizing your data (chose functions of features $x$ such that you get a Gaussian distribution i.e.:
- $f_1(x) = log(x +c) $
- $f(x)2 = x.^{1/2}$
- ...
- $f_8(x) = x.^{1/3}$
(check by plotting histograms) also Multivariate Gaussian models may be of value for new features that have some kind of correlation (e.g., see here).
A density detection algorithm could be applied which may better fit your needs than a neural network (a formula for which could be found in Wikipedia per above and in this publicly available article, it is also possible if this method is used to create the 3 subsets of your +- 27 000neg and +- 3000pos examples slightly differently, i.e. allow your training set to have say 60% of the negative examples (16 200) and it would have no positive examples, allow your cross validation set to have (5400 neg and 1500 pos) and your test set to have (5400 neg and 1500pos).
You can use your cross validation set to automatically choose your threshold and allows you to adjust your system and tweak it.
As measures of success consider different evaluation formats and counts (sometimes seeing how items were classified in different measures can help you visualise the problem more clearly):
Calculate your F score (as this may be a good second way to view how well your algorithm is doing since your data set is bias to negative examples the formula for which is simply: $2*((\text{precision}*\text{recall})/(\text{precision} + \text{recall}))$.
And finally use your test set to verify classifiers' ability without any tweaking based on these results.