First, I'd recommend starting with the sample data that is provided with the software. Most software distributions include example data that you can use to get familiar with the algorithm without dealing with data types and wrestling the data into the right format for the algorithm. Even if you are building an algorithm from scratch, you can start with the sample from a similar implementation and compare the performance.
Second, I'd recommend experimenting with synthetic data sets to get a feel for how the algorithm performs when you know how the data was generated and the signal to noise ratio.
In R, you can list all dataset in the currently installed packages with this command:
data(package = installed.packages()[, 1])
The R package mlbench has real datasets and can generate synthetic datasets that are useful for studying algorithm performance.
Python's scikit-learn has sample data and generates synthetic/toy dataset too.
SAS has training dataset available for download and the SPSS sample data is installed with the software at C:\Program Files\IBM\SPSS\Statistics\22\Samples
Lastly, I'd look at data in the wild. I'd compare the performance of different algorithms and tuning parameters on real data sets. This usually requires a lot more work because you will rarely find dataset with data types and structures that you can drop right into your algorithms.
For data in the wild, I'd recommend:
reddit's Dataset Archive
KDnugget's list