What sort of problems is backpropagation best suited to solving, and what are the best alternatives to backprop for solving those problems? I am developing a neuroscience-inspired training algorithm for  feed-forward neural networks. The natural benchmark for comparison is backpropagation. So I need to know to know what sort of classification problems backpropagation (and thus feed-forward NNs) is most suited for. Also, since I ultimately want to move beyond backprop, what non NN algorithms are the best alternatives to backprop, when it comes to the sort of problems that backprop is good at. 
Or, in other words, what problems should I benchmark a feed-forward network on, and what are the algorithms to beat for those problems?
 A: In general, feed-forward networks utilizing backpropagation are great for classification tasks in which you have a number of probabilistic cues which need to be integrated. They're obviously used for a great many other things, but this is an example of one driving many people to use them -- it's difficult to capture this sort of learning in a more Hebbian fashion.
In any case where you have multiple probabilistic cues which may be of limited, but above-chance use in classification, they can be integrated in a cognitively plausible manner (as in natural language acquisition by children) -- the integration of multiple cues, any one of which is a weak cue for classification in and of itself, can result in very robust classification performance.
For this sort of classification w/ multiple cues, AdaBoost would be an appropriate (and quite well-established) algorithm to use as a baseline.
An off-the-shelf support vector machine might rival AdaBoost's performance. If you're looking for baseline classifiers that people are already familiar with, you should definitely use an SVM for at least one of them. There are also approaches that combine AdaBoost with SVMs.
A: Edit
Originally this answer discussed alternate learning algorithms and topologies for Neural Nets.  After the edit, the answer is divided into three parts:

*

*Uses and problems with Backpropogation;

*Alternate Neural Network Training Schemes; and

*Alternates to Neural Networks (newly added).


Part 1
Backpropogation algorithms can help to solve problems where there is a discriminant that can help to separate the positive inputs from the negative inputs, often in networks that have no connections that loop between neurons in their topology (feed forward neural networks as described in your question).
Backpropogation is typically slow and it is not guaranteed to converge and may converge to a local maximum (that is, it will converge up to a point were you get results that are better than the results that can be obtained by changing the weights slightly).
Part 2
Alternate neural network training algorithms include using a Genetic Algorithm to evolve the weights and simulated annealing.  These may eliminate the problems related to slow convergence and problems with connections that loop between neurons.
Hebbian learning as used in recurrent neural networks, typically Hopfield networks, is another alternative.  Hebbian learning is robust and Hopfield proved that Hebbian learning will converge for Hopfield networks.
You might also consider comparing your neural network to a modular neural network, such as, for example a Committee of Machines made up of several neural networks each with their own training algorithms or an Associative Neural Network.
Such modular neural networks will require more space and time to work with, but ultimately may perform better if their topologies are designed well for the particular problem at hand.
Part 3
Popular alternatives to consider include Support Vector Machines (which are widely used in Fraud Analysis and Network Intrusion Detection amongst other areas).
Genetic Algorithms and Simulated Annealing (as stand-alone algorithms and not in conjunction with Neural Nets).
A final group of algorithms to consider are Swarm Algorithms such as Ant Colony Optimisation.  Some algorithms in this group such as continuous orthogonal ant colony optimisation can be very powerful for solving some problems.
You might consider using some sort of clustering algorithms.  Depending on your problems, K-Medoids or Density Based Clustering might be useful.
Depending on your problem, Machine Vision algorithms might also be useful.  For example, the algorithm by Milanfar, http://users.soe.ucsc.edu/~milanfar/research/computer-vision.html, based on local regression kernels might also be useful.
The exact algorithms to use might depend on the exact problem that you are facing.  Some algorithms will work really well in some instances but really badly in another.  Personally I do not believe that there is a single algorithm that is perfect for solving everything (if there were we would be using it), so you would need to decide on the algorithm to benchmark against depending on the scenario in which you will be using it.
