Can arbitrary precision calculations be useful for machine learning? Regarding machine learning methods but dealing with arbitrary floating-point precision. It sounds cool to me but I am unsure if this would be of any use...
Did anyone ever encountered cases where, for example, a logistic regression task needs to be carried out with an extremely high precision? Can it make sense? I am looking for opinions from ML experts.
Edit: If not arbitrary high, maybe quadruple precision could be useful?
 A: Yes, precision can be problematic on multiple fronts. First, regression itself generally approaches a flat region, like the bottom of a parabola, where the minimum loss function of the regression is located. This typically halves the number of significant figures in the loss function and may reduce the precision of any parameters of a model even more than that. Second, in order to calculate some transcendental functions one may need much higher precision for the calculation process itself than are available for the functional value when that process is completed. See https://blogs.ubc.ca/infiniteseriesmodule/units/unit-3-power-series/taylor-series/maclaurin-expansion-of-sinx/ For example, look at how the intermediate terms of the series expansion of sin(x) achieve large magnitudes before convergence for the sin(12 radians)

This shows that although sin(x) is bounded above and below by $\pm1$, the individual terms are sometimes $\pm20000$, so if we are not careful, the absolute error from those terms could be greater than 1. Now it is true that one can, rather than take the sine of 12, transform that request so that the calculation is performed in the region of principal sine, for which the precision problem is mitigated (but not eliminated), however, the point here is that to guarantee any particular precision for a function, more precision may be necessary during the calculation than is returned when that functional answer is produced.
Edit: The additional OP edit question is whether quadruple precision is enough. The correct answer is sometimes. To see what precision is needed some type of error propagation analysis is needed. Also see propagation of uncertainty and the delta method. That will tell you what the function for fitting needs for precision, if you know what that is. Then one has to account for the precision loss for the loss functions used during regression, and most software allows you to specify what that is. One should then augment the data calculation precision to include both the functional precision loss and the regression precision loss.
In the sine example above, one can find the maximum absolute term magnitude in low precision, let's call that  $\Delta$, and then set the precision at $D+\log_{10}(\Delta)$, where $D$ is the precision desired, and only then calculate each term value at the higher precision for later summation. Some software routines augment precision automatically during summation, others do not, and if not, then the error propagation from summation would be added to the precision request for $\sin(x)$.
One final suggestion. Calculating what precision is needed for which problem can be daunting and in some cases for which the algorithms are not well characterized, e.g., some machine learning routines, are not reasonably achievable. In that case, one could proceed heuristically by increasing precision until it no longer makes a difference to the precision desired in the answer. But, don't be surprised if that turns out to be a larger number of significant figures than one would guess without doing such a test.
A: No, it is almost never a problem. First of all, there's a measurement error—even physicists account for it, while the rest of us rarely can be lucky enough for as precise measurements as theirs. Second, you are dealing with sampled data, so there is error due to sampling. Finally, we have all kinds of biases and noise in the data. In the end, we are usually far from having precise data, so we don’t need algorithms more precise than the data itself.
More than this, there is research showing that you can train neural networks with low (8-bit, 2-bit) precision without performance drop. Some argue that this might even have a regularizing effect. It can probably be extended to some degree to other models.
