How to make sure that a machine learning algorithm's implementation is correct? Say there is a machine learning algorithm (e.g. classification) that is well known and implemented by the original creators of the algorithm. Yet all you have is the ability to use the algorithm but not see the source code.
Now you want to implement this algorithm and then check if your implementation is correct or not. Since you can't use the same initialization values to the parameters (because you don't have access to the source code of the original implementation) then you can't expect to get exactly the same results. Say that the algorithm gives as outputs probabilities, then your method could give a slightly different result from the original method.
It's true that your implementation could give the correct classification results as with the original implementation but the probabilities could be different. 
Given such a scenario, how would you judge that your implementation is correct?
This exact situation happened with a friend whose implementation gave the correct classification results but after all he found a serious bug in his code!
So is there a way to test and validate the implementation of a machine learning algorithm in general? Regardless of type of the algorithm (classification, regression ... etc).
The only problem I can see is not being able to use the same initial values for the parameters. Then maybe there is some kind of a statistical test that can test the results with respect to the initial values of the parameters somehow.
 A: There are various metrics for algorithm's performance (precision, recall, f1, etc.). I'd start by searching for a paper by the algorithm's authors where they mention what kind of data they have tested their algorithm on, what are the results (what metric do they mention) of their algorithm on that data. Then I'd search for the same or similar data, run my implementation on it and compare the results with theirs
A: On the Generalized Testing of Machine Learning Algorithms:
Yes, if there is a known working method, comparing your result to that method over all possible parameters will guarantee that your program is also a known working method.  This is usually impossible and always pointless because there already is a known working method.
If there isn't a known working method, then in general no, as a counter point consider this "code" that calculates regression coefficients: 
$\hat{\beta} = (X'X)^{-1}X'y + \delta$
where $\delta = 1000$ when $y[1] = \pi$ else 0. 
This implementation is right almost all of the time (technically right a.s., but not under IEEE 754), and it is computationally intractable to find it's error. 
On the Implementations of Methods:
Standard practice is the same as in all software development small test cases, and constant validation against something known.  E.g. if the model under certain parameters has a known closed form solution or is equivalent to another method check that.  
Also note that papers's aren't perfect, in most lit reviews I do I usually find a few typo's.  Some of these typos actually would cause incorrect results, so always double check your sources.  
There is a reason emails are on journal papers (HINT: it's so you can contact the authors).  Also note that author's don't generally bite if you are nice to them.  Just be courteous, and show that you have done some work.  But don't expect them to step through your broken code to find a bug.
If you have the case where an author won't release the code, or won't respond to you, then don't use that method, more than likely the quality of the code was "good enough to publish" but that's about it.  There certainly is no shortage of machine learning algorithms out there.  It's also worth while checking who has cited the paper in question and see if they have some code.
A: Do you know what "correct" is for your situation?  Determine what correct means then test your implementation against your data and see how close you are.
Unless your test coverage is 100% you'll never know if you have a nasty edge case bug in your code.  You wouldn't if you had the initial parameters of the algorithm you are modeling unless your test touched the code where the bug lives.
