changeset 31556:9978ef12aea3 stable

avoid overflow due to abs(min_val) in signed int64 multiplication (bug #63416) * oct-inttypes.cc (octave_int_arith_base<int64_t, true>::mul_internal): Avoid calling abs on min_val.
author John W. Eaton <jwe@octave.org>
date Sat, 26 Nov 2022 17:08:30 -0500
parents 9d0d37acafbe
children 2fb47f515612 69c8979f8b2c
files liboctave/util/oct-inttypes.cc
diffstat 1 files changed, 8 insertions(+), 0 deletions(-) [+]
line wrap: on
line diff
--- a/liboctave/util/oct-inttypes.cc	Sat Nov 26 17:05:21 2022 -0500
+++ b/liboctave/util/oct-inttypes.cc	Sat Nov 26 17:08:30 2022 -0500
@@ -378,6 +378,14 @@
   // Essentially, what we do is compute sign, multiply absolute values
   // (as above) and impose the sign.
 
+  // But first, avoid overflow in computation of abs (min_val ()).
+
+  if (x == min_val ())
+    return y == 0 ? 0 : (y < 0 ? max_val () : min_val ());
+
+  if (y == min_val ())
+    return x == 0 ? 0 : (x < 0 ? max_val () : min_val ());
+
   uint64_t usx = octave_int_abs (x);
   uint64_t usy = octave_int_abs (y);
   bool positive = (x < 0) == (y < 0);