stepcompress: Always return 0 on negative number in safe_sqrt()

sqrt() of a negative number in the C code returns NaN.  This value
results in behavior that is difficult to debug.  Always return 0.0
instead as this results in better behavior that is easier to track
down.  Also, on some code paths, safe_sqrt is called on numbers that
are multiplied by very large amounts (eg, 16000000**2) and thus
distinguishing between large and small negative numbers is difficult.
For now, report in the log if the value is below -0.001.

Signed-off-by: Kevin O'Connor <kevin@koconnor.net>
This commit is contained in:
Kevin O'Connor 2017-03-22 10:46:59 -04:00
parent 9bf73cd72d
commit 8d92c898ee
1 changed files with 5 additions and 5 deletions

View File

@ -281,11 +281,11 @@ check_line(struct stepcompress *sc, struct step_move move)
static double static double
_safe_sqrt(double v) _safe_sqrt(double v)
{ {
if (v > -0.001) // Due to floating point truncation, it's possible to get a small
// Due to floating point truncation, it's possible to get a // negative number - treat it as zero.
// small negative number - treat it as zero. if (v < -0.001)
return 0.; errorf("safe_sqrt of %.9f", v);
return sqrt(v); return 0.;
} }
static inline double safe_sqrt(double v) { static inline double safe_sqrt(double v) {
return likely(v >= 0.) ? sqrt(v) : _safe_sqrt(v); return likely(v >= 0.) ? sqrt(v) : _safe_sqrt(v);