Mercurial > octave

comparison doc/interpreter/numbers.txi @ 31077:2dee06f4635c stable

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc: Rewrite section on automatic type conversions (bug #62283) numbers.txi: Change language about promotion and demotion to conversion. octave.texi: Update new section title.
author Arun Giridhar <arungiridhar@gmail.com>
date Mon, 06 Jun 2022 15:37:26 -0400
parents 43974344fe19
children 96f751f8392c
comparison
equal deleted inserted replaced
31076:7ea420f2c722 31077:2dee06f4635c
100 * Ranges:: 100 * Ranges::
101 * Single Precision Data Types:: 101 * Single Precision Data Types::
102 * Integer Data Types:: 102 * Integer Data Types::
103 * Bit Manipulations:: 103 * Bit Manipulations::
104 * Logical Values:: 104 * Logical Values::
105 * Promotion and Demotion of Data Types:: 105 * Automatic Conversion of Data Types::
106 * Predicates for Numeric Objects:: 106 * Predicates for Numeric Objects::
107 @end menu 107 @end menu
108 108
109 @node Matrices 109 @node Matrices
110 @section Matrices 110 @section Matrices
707 707
708 @DOCSTRING(true) 708 @DOCSTRING(true)
709 709
710 @DOCSTRING(false) 710 @DOCSTRING(false)
711 711
712 @node Promotion and Demotion of Data Types 712 @node Automatic Conversion of Data Types
713 @section Promotion and Demotion of Data Types 713 @section Automatic Conversion of Data Types
714 714
715 Many operators and functions can work with mixed data types. For example, 715 Many operators and functions can work with mixed data types. For example,
716 716
717 @example 717 @example
718 @group 718 @group
719 uint8 (1) + 1 719 uint8 (1) + 1
720 @result{} 2 720 @result{} 2
721 @end group 721 @end group
722 @end example 722
723
724 @noindent
725 where the above operator works with an 8-bit integer and a double precision
726 value and returns an 8-bit integer value. Note that the type is demoted
727 to an 8-bit integer, rather than promoted to a double precision value as
728 might be expected. The reason is that if Octave promoted values in
729 expressions like the above with all numerical constants would need to be
730 explicitly cast to the appropriate data type like
731
732 @example
733 @group
734 uint8 (1) + uint8 (1)
735 @result{} 2
736 @end group
737 @end example
738
739 @noindent
740 which becomes difficult for the user to apply uniformly and might allow
741 hard to find bugs to be introduced. The same applies to single precision
742 values where a mixed operation such as
743
744 @example
745 @group 723 @group
746 single (1) + 1 724 single (1) + 1
747 @result{} 2 725 @result{} 2
748 @end group 726 @end group
749 @end example 727
750 728 @group
751 @noindent 729 min (single (1), 0)
752 returns a single precision value. The mixed operations that are valid 730 @result{} 0
753 and their returned data types are 731 @end group
732 @end example
733
734 @noindent
735 where the results are respectively of types uint8, single, and single
736 respectively. This is done for Matlab compatibility. Valid mixed operations
737 are defined as follows:
754 738
755 @multitable @columnfractions .2 .3 .3 .2 739 @multitable @columnfractions .2 .3 .3 .2
756 @headitem @tab Mixed Operation @tab Result @tab 740 @headitem @tab Mixed Operation @tab Result @tab
757 @item @tab double OP single @tab single @tab 741 @item @tab double OP single @tab single @tab
758 @item @tab double OP integer @tab integer @tab 742 @item @tab double OP integer @tab integer @tab
761 @item @tab single OP integer @tab integer @tab 745 @item @tab single OP integer @tab integer @tab
762 @item @tab single OP char @tab single @tab 746 @item @tab single OP char @tab single @tab
763 @item @tab single OP logical @tab single @tab 747 @item @tab single OP logical @tab single @tab
764 @end multitable 748 @end multitable
765 749
766 The same logic applies to functions with mixed arguments such as 750 When functions expect a double but are passed other types, automatic
767 751 conversion is function-dependent:
768 @example
769 @group
770 min (single (1), 0)
771 @result{} 0
772 @end group
773 @end example
774
775 @noindent
776 where the returned value is single precision.
777
778 Many functions and operators will also promote integer or logical types to
779 double, or single to double, especially if they take only one argument.
780 752
781 @example 753 @example
782 @group 754 @group
783 a = det (int8 ([1 2; 3 4])) 755 a = det (int8 ([1 2; 3 4]))
784 @result{} a = -2 756 @result{} a = -2
785 class (a) 757 class (a)
786 @result{} double 758 @result{} double
787 @end group 759 @end group
788 @end example 760
789
790 But there are also exceptions for promoting to double, especially if the
791 function or operator in question can take multiple arguments.
792
793 @example
794 @group 761 @group
795 a = eig (int8 ([1 2; 3 4])) 762 a = eig (int8 ([1 2; 3 4]))
796 @result{} error: eig: wrong type argument 'int8 matrix' 763 @result{} error: eig: wrong type argument 'int8 matrix'
797 @end group 764 @end group
798 @end example 765 @end example
799 766
800 When the two operands are both integers but of different widths, then the 767 When two operands are both integers but of different widths, then some cases
801 behavior depends on the operator or the function in question. For some 768 convert them to the wider bitwidth, and other cases throw an error:
802 operators and functions, narrow-bitwidth operands are promoted to a wider
803 bitwidth:
804 769
805 @example 770 @example
806 @group 771 @group
807 a = min (int8 (100), int16 (200)) 772 a = min (int8 (100), int16 (200))
808 @result{} 100 773 @result{} 100
809 class (a) 774 class (a)
810 @result{} int16 775 @result{} int16
811 @end group 776 @end group
812 @end example 777
813
814 However, not all functions or operators will accept integer operands of
815 differing types:
816
817 @example
818 @group 778 @group
819 int8 (100) + int16 (200) 779 int8 (100) + int16 (200)
820 @result{} error: binary operator '+' not implemented 780 @result{} error: binary operator '+' not implemented
821 for 'int8 scalar' by 'int16 scalar' operations 781 for 'int8 scalar' by 'int16 scalar' operations
822 @end group 782 @end group
823 @end example 783 @end example
824 784
825 Further, in most cases, both operands need to be signed or both need to be 785 For two integer operands, they typically need to both be signed or both be
826 unsigned. Mixing signed and unsigned usually causes an error, even if they 786 unsigned. Mixing signed and unsigned usually causes an error, even if they
827 are of the same bitwidth. 787 are of the same bitwidth.
828 788
829 @example 789 @example
830 @group 790 @group
831 min (int8 (100), uint16 (200)) 791 min (int16 (100), uint16 (200))
832 @result{} error: min: cannot compute min (int8 scalar, uint16 scalar) 792 @result{} error: min: cannot compute min (int16 scalar, uint16 scalar)
833 @end group 793 @end group
834 @end example 794 @end example
835 795
836 In the case of mixed type indexed assignments, the type is not 796 In the case of mixed type indexed assignments, the type is not
837 changed. For example, 797 changed. For example,