To supplement Dikran Marsupial's answer, the following is an articulation of a process I use personally. This is from the perspective of coding machine learning algorithms for research, rather than from a full-blown production level software engineering perspective.
Primarily, I've often found it useful to see the process of coding a machine learning algorithm from scratch as instantiating a mathematical idea:
Mathematical, pre-algorithmic stage.
Usually in machine learning, you use some kind of mathematical or formal principle to guide the development of your algorithm. It may be statistical e.g. maximum likelihood estimation, and most likely, involve a degree of optimisation. Usually, you will need to do a fair amount of mathematical derivation work, until you have what you need.
Personally, I've found that one needs both a solid under-the-hood grasp of the mathematical principles operating here, as well as absolute security in the correctness of one's derivations. Any haziness or uncertainty at this stage will compound during implementation, when one additionally has to worry about debugging. Before proceeding to the next stage, I will often use a symbolic computing package or computer algebra system, such as Mathematica, to check all my derivations, because even a single error at this stage can lead to the extremely undesirable situation of having to search for derivation and/or debugging errors.
After I've checked that my mathematical derivations are watertight, with no errors, and I'm at a level where I can specify and sketch out my algorithm on paper, I proceed to drafting formal pseudocode. Similar to the above poster, I find that forcing oneself to type this up using
algorithm2e in LaTeX helps in that it commits you in a way working solely with pen and paper doesn't.
During this pseudo-code drafting stage, I start thinking about more implementational concerns, such as adapting my derivations to be as vectorised as possible, i.e. using as few loops as possible, or making use of linear algebra techniques as far as possible. However, the implementational concerns extend only as far as complying with basic advice and standard principles of algorithm design, but not as far as optimising the code.
Implement the pseudocode in code. The clearer your pseudocode, and the more consistent your notation, the easier this will be. Something you will probably pick for up for yourself is that it pays enormous dividends, as the above poster has stated, to ensure you are writing modular code. Split the code writing into little chunks, which you can individually test.
In most cases, there will almost always be very basic sanity checks you can do. For example, if your code uses gradients, then you can use finite differencing to assess this has correctly been computed. If you are implementing an EM algorithm, your log-likelihood will monotonically increase, so any decreases are most likely coding errors, provided your derivations are correct.
I don't want to say much about this, except that it can get deep very quickly. At this stage, you time your code, or use a whole host of numerical linear algebra, matrix computation and convex optimisation tricks to squeeze as many performance speed-ups as possible.
Concerning the use of tools for debugging, I don't know much about the debugging tools in R as it's something I'm only getting to grips with. However, it looks like the RStudio debugger is similar to that of
pdb in Python.
In the case of Python, the PyCharm IDE is one example where I've found that tools can vastly improve the debugging process. You can walk through each line of your code as it runs, and step in, step out, do side by side evaluations, as well as keep track of all variable states at once. I personally find that the visual means of accessing all variable states at once is far superior to using doing manual command line queries using
pdb. Whilst I wish I knew this earlier, I emphasise this is for consideration only, and I have no affiliation with JetBrains whatsoever.
Some final things. Like most things in life, you get better through practice. There are lots of papers accompanied with code at paperswithcode.com; and there is also an ML paper reproducibility challenge also.
Lastly, if you ever find that debugging is starting to grind on your soul somewhat, I've found it helps to think of the days when people used to use punch cards to implement algorithms...