stepcompress: Fix rounding error when interval=0
In the rare case where the step generation code finds an interval of zero, it was possible for an integer division of a negative number to cause an incorrect result. Fix that by using the idiv_up() and idiv_down() helpers. Signed-off-by: Kevin O'Connor <kevin@koconnor.net>
This commit is contained in:
parent
502e83725b
commit
5ca132eafa
|
@ -115,9 +115,9 @@ compress_bisect_add(struct stepcompress *sc)
|
|||
int32_t nextaddfactor = nextcount*(nextcount-1)/2;
|
||||
int32_t c = add*nextaddfactor;
|
||||
if (nextmininterval*nextcount < nextpoint.minp - c)
|
||||
nextmininterval = DIV_ROUND_UP(nextpoint.minp - c, nextcount);
|
||||
nextmininterval = idiv_up(nextpoint.minp - c, nextcount);
|
||||
if (nextmaxinterval*nextcount > nextpoint.maxp - c)
|
||||
nextmaxinterval = (nextpoint.maxp - c) / nextcount;
|
||||
nextmaxinterval = idiv_down(nextpoint.maxp - c, nextcount);
|
||||
if (nextmininterval > nextmaxinterval)
|
||||
break;
|
||||
interval = nextmaxinterval;
|
||||
|
|
Loading…
Reference in New Issue