Mercurial > octave
changeset 33550:f852364698ed stable
Prevent OOM crash / segfault in sort() when dim = inf (bug #65712)
This is a cherry-picking of the following patches from default to stable
to prevent a crash that occurs with `sort ([1 2; 3 4], inf)`:
8f73f8534d42, 47c798415504, and a12d26fababc.
* Array-base.cc: Return early when sort dimension exceeds array dimension,
before a temporary array is constructed.
| author | Arun Giridhar <arungiridhar@gmail.com> |
|---|---|
| date | Wed, 08 May 2024 14:52:16 -0400 |
| parents | 8e555ddce800 |
| children | 4495e4a23aa6 6d7e7a2faf4b |
| files | liboctave/array/Array-base.cc |
| diffstat | 1 files changed, 12 insertions(+), 6 deletions(-) [+] |
line wrap: on
line diff
--- a/liboctave/array/Array-base.cc Fri May 03 01:54:53 2024 -0400 +++ b/liboctave/array/Array-base.cc Wed May 08 14:52:16 2024 -0400 @@ -1783,16 +1783,22 @@ if (dim < 0) (*current_liboctave_error_handler) ("sort: invalid dimension"); + if (numel () < 1) // input array is empty, + return *this; + + if (dim >= ndims ()) + { + // The dimension to sort along exceeds the array's dimensions, + // ==> array is already trivially sorted in such higher dimensions, + // ==> return early. + + return *this; + } + Array<T, Alloc> m (dims ()); dim_vector dv = m.dims (); - if (m.numel () < 1) - return m; - - if (dim >= dv.ndims ()) - dv.resize (dim+1, 1); - octave_idx_type ns = dv(dim); octave_idx_type iter = dv.numel () / ns; octave_idx_type stride = 1;