Finding best fit curve with unknown power If I was to build a program to estimate the best curve fit of type $a * x^b$ where a and b are the parameters I'm optimizing, what would be my go to methods? 
I know I can use ordinary least squares when b is known. Is there a similar closed form-method for this type of problem, or would I be left with only iterative methods like gradient descent?
 A: Fitting $\quad y=ax^b$
Since you don't specify the criteria of fitting they are an infinity of different solutions. For example, the solution will not be the same if you are looking for the least mean square error, or the least mean absolute error, or the least mean relative error, or etc.
So if any one of them is sufficient for you, the simplest way is to fit the related logarithmic function. Change of variables :
$$\begin{cases} Y=\ln(y)\\ X=\ln(x) \end{cases}$$
The function to fit is linear : 
$$Y=bX+c$$
Proceed to a linear regression which will give you $b$ and $c$, then $a= e^c$.
If a criteria of fitting is specified, you need to proceed to a non-linear regression : http://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html
NOTE : 
The problem is slightly more complicated if the function to fit includes three parameters instead of two :
$$y=ax^b+c$$
The above change of variables doesn't transform the function to a linear function. A non-linear regression (such as described in the above reference and link) is necessary. 
Again if no criteria of fitting is specified, a much simpler method (not iterative, no initial guess) is shown page 17 in https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales .
