I am estimating a non-linear correspondence between 3D and 2D points using
scipy.optimize.least_squares. I found that setting the method to be Trust Region Reflective algorithm and soft L1 loss gives the smallest error
When I try to project 3D point to the 2D plane, I get a strange behaviour as can be seen in the images below. As you can see, all the projected points translate to the to the side as Z increases.
Can I overcome this behavior somehow using the SciPy library?