From 1850abe1c73f64e688c5277af7b96be46017491f Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Wed, 15 Dec 2021 12:07:50 -0500 Subject: [PATCH 1/6] Transform broadcasting.md to .rst --- spec/API_specification/boradcasting.rst | 118 ++++++++++++++++++++++++ spec/API_specification/broadcasting.md | 116 ----------------------- 2 files changed, 118 insertions(+), 116 deletions(-) create mode 100644 spec/API_specification/boradcasting.rst delete mode 100644 spec/API_specification/broadcasting.md diff --git a/spec/API_specification/boradcasting.rst b/spec/API_specification/boradcasting.rst new file mode 100644 index 000000000..3692691d2 --- /dev/null +++ b/spec/API_specification/boradcasting.rst @@ -0,0 +1,118 @@ +Broadcasting +============ + + Array API specification for broadcasting semantics. + +Overview +-------- + +**Broadcasting** refers to the automatic (implicit) expansion of array dimensions to be of equal sizes without copying array data for the purpose of making arrays with different shapes have compatible shapes for element-wise operations. + +Broadcasting facilitates user ergonomics by encouraging users to avoid unnecessary copying of array data and can **potentially** enable more memory-efficient element-wise operations through vectorization, reduced memory consumption, and cache locality. + +Algorithm +--------- + +Given an element-wise operation involving two compatible arrays, an array having a singleton dimension (i.e., a dimension whose size is one) is broadcast (i.e., virtually repeated) across an array having a corresponding non-singleton dimension. + +If two arrays are of unequal rank, the array having a lower rank is promoted to a higher rank by (virtually) prepending singleton dimensions until the number of dimensions matches that of the array having a higher rank. + +The results of the element-wise operation must be stored in an array having a shape determined by the following algorithm. + +1. Let ``A`` and ``B`` both be arrays. + +1. Let ``shape1`` be a tuple describing the shape of array ``A``. + +1. Let ``shape2`` be a tuple describing the shape of array ``B``. + +1. Let ``N1`` be the number of dimensions of array ``A`` (i.e., the result of ``len(shape1)``). + +1. Let ``N2`` be the number of dimensions of array ``B`` (i.e., the result of ``len(shape2)``). + +1. Let ``N`` be the maximum value of ``N1`` and ``N2`` (i.e., the result of ``max(N1, N2)``). + +1. Let ``shape`` be a temporary list of length ``N`` for storing the shape of the result array. + +1. Let ``i`` be ``N-1``. + +1. Repeat, while ``i >= 0`` + + 1. Let ``n1`` be ``N1 - N + i``. + + 1. If ``n1 >= 0``, let ``d1`` be the size of dimension ``n1`` for array ``A`` (i.e., the result of ``shape1[n1]``); else, let ``d1`` be ``1``. + + 1. Let ``n2`` be ``N2 - N + i``. + + 1. If ``n2 >= 0``, let ``d2`` be the size of dimension ``n2`` for array ``B`` (i.e., the result of ``shape2[n2]``); else, let ``d2`` be ``1``. + + 1. If ``d1 == 1``, then + + - set the ``i``th element of ``shape`` to ``d2``. + + 1. Else, if ``d2 == 1``, then + + - set the ``i``th element of ``shape`` to ``d1``. + + 1. Else, if ``d1 == d2``, then + + - set the ``i``th element of ``shape`` to ``d1``. + + 1. Else, throw an exception. + + 1. Set ``i`` to ``i-1``. + +1. Let ``tuple(shape)`` be the shape of the result array. + +Examples +~~~~~~~~ + +The following examples demonstrate the application of the broadcasting algorithm for two compatible arrays. + +:: + A (4d array): 8 x 1 x 6 x 1 + B (3d array): 7 x 1 x 5 + --------------------------------- + Result (4d array): 8 x 7 x 6 x 5 + + A (2d array): 5 x 4 + B (1d array): 1 + ------------------------- + Result (2d array): 5 x 4 + + A (2d array): 5 x 4 + B (1d array): 4 + ------------------------- + Result (2d array): 5 x 4 + + A (3d array): 15 x 3 x 5 + B (3d array): 15 x 1 x 5 + ------------------------------ + Result (3d array): 15 x 3 x 5 + + A (3d array): 15 x 3 x 5 + B (2d array): 3 x 5 + ------------------------------ + Result (3d array): 15 x 3 x 5 + + A (3d array): 15 x 3 x 5 + B (2d array): 3 x 1 + ------------------------------ + Result (3d array): 15 x 3 x 5 + + +The following examples demonstrate array shapes which do **not** broadcast. + +:: + A (1d array): 3 + B (1d array): 4 # dimension does not match + + A (2d array): 2 x 1 + B (3d array): 8 x 4 x 3 # second dimension does not match + + A (3d array): 15 x 3 x 5 + B (2d array): 15 x 3 # singleton dimensions can only be prepended, not appended + +In-place Semantics +------------------ + +As implied by the broadcasting algorithm, in-place element-wise operations must not change the shape of the in-place array as a result of broadcasting. \ No newline at end of file diff --git a/spec/API_specification/broadcasting.md b/spec/API_specification/broadcasting.md deleted file mode 100644 index 34e90cd0b..000000000 --- a/spec/API_specification/broadcasting.md +++ /dev/null @@ -1,116 +0,0 @@ -(broadcasting)= - -# Broadcasting - -> Array API specification for broadcasting semantics. - -## Overview - -**Broadcasting** refers to the automatic (implicit) expansion of array dimensions to be of equal sizes without copying array data for the purpose of making arrays with different shapes have compatible shapes for element-wise operations. - -Broadcasting facilitates user ergonomics by encouraging users to avoid unnecessary copying of array data and can **potentially** enable more memory-efficient element-wise operations through vectorization, reduced memory consumption, and cache locality. - -## Algorithm - -Given an element-wise operation involving two compatible arrays, an array having a singleton dimension (i.e., a dimension whose size is one) is broadcast (i.e., virtually repeated) across an array having a corresponding non-singleton dimension. - -If two arrays are of unequal rank, the array having a lower rank is promoted to a higher rank by (virtually) prepending singleton dimensions until the number of dimensions matches that of the array having a higher rank. - -The results of the element-wise operation must be stored in an array having a shape determined by the following algorithm. - -1. Let `A` and `B` both be arrays. - -1. Let `shape1` be a tuple describing the shape of array `A`. - -1. Let `shape2` be a tuple describing the shape of array `B`. - -1. Let `N1` be the number of dimensions of array `A` (i.e., the result of `len(shape1)`). - -1. Let `N2` be the number of dimensions of array `B` (i.e., the result of `len(shape2)`). - -1. Let `N` be the maximum value of `N1` and `N2` (i.e., the result of `max(N1, N2)`). - -1. Let `shape` be a temporary list of length `N` for storing the shape of the result array. - -1. Let `i` be `N-1`. - -1. Repeat, while `i >= 0` - - 1. Let `n1` be `N1 - N + i`. - - 1. If `n1 >= 0`, let `d1` be the size of dimension `n1` for array `A` (i.e., the result of `shape1[n1]`); else, let `d1` be `1`. - - 1. Let `n2` be `N2 - N + i`. - - 1. If `n2 >= 0`, let `d2` be the size of dimension `n2` for array `B` (i.e., the result of `shape2[n2]`); else, let `d2` be `1`. - - 1. If `d1 == 1`, then - - - set the `i`th element of `shape` to `d2`. - - 1. Else, if `d2 == 1`, then - - - set the `i`th element of `shape` to `d1`. - - 1. Else, if `d1 == d2`, then - - - set the `i`th element of `shape` to `d1`. - - 1. Else, throw an exception. - - 1. Set `i` to `i-1`. - -1. Let `tuple(shape)` be the shape of the result array. - -### Examples - -The following examples demonstrate the application of the broadcasting algorithm for two compatible arrays. - -```text -A (4d array): 8 x 1 x 6 x 1 -B (3d array): 7 x 1 x 5 ---------------------------------- -Result (4d array): 8 x 7 x 6 x 5 - -A (2d array): 5 x 4 -B (1d array): 1 -------------------------- -Result (2d array): 5 x 4 - -A (2d array): 5 x 4 -B (1d array): 4 -------------------------- -Result (2d array): 5 x 4 - -A (3d array): 15 x 3 x 5 -B (3d array): 15 x 1 x 5 ------------------------------- -Result (3d array): 15 x 3 x 5 - -A (3d array): 15 x 3 x 5 -B (2d array): 3 x 5 ------------------------------- -Result (3d array): 15 x 3 x 5 - -A (3d array): 15 x 3 x 5 -B (2d array): 3 x 1 ------------------------------- -Result (3d array): 15 x 3 x 5 -``` - -The following examples demonstrate array shapes which do **not** broadcast. - -```text -A (1d array): 3 -B (1d array): 4 # dimension does not match - -A (2d array): 2 x 1 -B (3d array): 8 x 4 x 3 # second dimension does not match - -A (3d array): 15 x 3 x 5 -B (2d array): 15 x 3 # singleton dimensions can only be prepended, not appended -``` - -## In-place Semantics - -As implied by the broadcasting algorithm, in-place element-wise operations must not change the shape of the in-place array as a result of broadcasting. \ No newline at end of file From 13a0dd532951be651f986ab1a210869512687bde Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Wed, 15 Dec 2021 12:37:06 -0500 Subject: [PATCH 2/6] Fix typo and warnings --- spec/API_specification/{boradcasting.rst => broadcasting.rst} | 2 ++ 1 file changed, 2 insertions(+) rename spec/API_specification/{boradcasting.rst => broadcasting.rst} (99%) diff --git a/spec/API_specification/boradcasting.rst b/spec/API_specification/broadcasting.rst similarity index 99% rename from spec/API_specification/boradcasting.rst rename to spec/API_specification/broadcasting.rst index 3692691d2..b930aaa12 100644 --- a/spec/API_specification/boradcasting.rst +++ b/spec/API_specification/broadcasting.rst @@ -1,3 +1,5 @@ +.. _broadcasting: + Broadcasting ============ From 115b82caad9a2922ebec487581ca0d61c5b64d21 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Gacha Date: Tue, 21 Dec 2021 19:20:29 -0500 Subject: [PATCH 3/6] Fix ordered lists --- spec/API_specification/broadcasting.rst | 40 ++++++++++++------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/spec/API_specification/broadcasting.rst b/spec/API_specification/broadcasting.rst index b930aaa12..8b55e5e70 100644 --- a/spec/API_specification/broadcasting.rst +++ b/spec/API_specification/broadcasting.rst @@ -21,49 +21,49 @@ If two arrays are of unequal rank, the array having a lower rank is promoted to The results of the element-wise operation must be stored in an array having a shape determined by the following algorithm. -1. Let ``A`` and ``B`` both be arrays. +#. Let ``A`` and ``B`` both be arrays. -1. Let ``shape1`` be a tuple describing the shape of array ``A``. +#. Let ``shape1`` be a tuple describing the shape of array ``A``. -1. Let ``shape2`` be a tuple describing the shape of array ``B``. +#. Let ``shape2`` be a tuple describing the shape of array ``B``. -1. Let ``N1`` be the number of dimensions of array ``A`` (i.e., the result of ``len(shape1)``). +#. Let ``N1`` be the number of dimensions of array ``A`` (i.e., the result of ``len(shape1)``). -1. Let ``N2`` be the number of dimensions of array ``B`` (i.e., the result of ``len(shape2)``). +#. Let ``N2`` be the number of dimensions of array ``B`` (i.e., the result of ``len(shape2)``). -1. Let ``N`` be the maximum value of ``N1`` and ``N2`` (i.e., the result of ``max(N1, N2)``). +#. Let ``N`` be the maximum value of ``N1`` and ``N2`` (i.e., the result of ``max(N1, N2)``). -1. Let ``shape`` be a temporary list of length ``N`` for storing the shape of the result array. +#. Let ``shape`` be a temporary list of length ``N`` for storing the shape of the result array. -1. Let ``i`` be ``N-1``. +#. Let ``i`` be ``N-1``. -1. Repeat, while ``i >= 0`` +#. Repeat, while ``i >= 0`` - 1. Let ``n1`` be ``N1 - N + i``. + #. Let ``n1`` be ``N1 - N + i``. - 1. If ``n1 >= 0``, let ``d1`` be the size of dimension ``n1`` for array ``A`` (i.e., the result of ``shape1[n1]``); else, let ``d1`` be ``1``. + #. If ``n1 >= 0``, let ``d1`` be the size of dimension ``n1`` for array ``A`` (i.e., the result of ``shape1[n1]``); else, let ``d1`` be ``1``. - 1. Let ``n2`` be ``N2 - N + i``. + #. Let ``n2`` be ``N2 - N + i``. - 1. If ``n2 >= 0``, let ``d2`` be the size of dimension ``n2`` for array ``B`` (i.e., the result of ``shape2[n2]``); else, let ``d2`` be ``1``. + #. If ``n2 >= 0``, let ``d2`` be the size of dimension ``n2`` for array ``B`` (i.e., the result of ``shape2[n2]``); else, let ``d2`` be ``1``. - 1. If ``d1 == 1``, then + #. If ``d1 == 1``, then - set the ``i``th element of ``shape`` to ``d2``. - 1. Else, if ``d2 == 1``, then + #. Else, if ``d2 == 1``, then - set the ``i``th element of ``shape`` to ``d1``. - 1. Else, if ``d1 == d2``, then + #. Else, if ``d1 == d2``, then - set the ``i``th element of ``shape`` to ``d1``. - 1. Else, throw an exception. + #. Else, throw an exception. - 1. Set ``i`` to ``i-1``. + #. Set ``i`` to ``i-1``. -1. Let ``tuple(shape)`` be the shape of the result array. +#. Let ``tuple(shape)`` be the shape of the result array. Examples ~~~~~~~~ @@ -117,4 +117,4 @@ The following examples demonstrate array shapes which do **not** broadcast. In-place Semantics ------------------ -As implied by the broadcasting algorithm, in-place element-wise operations must not change the shape of the in-place array as a result of broadcasting. \ No newline at end of file +As implied by the broadcasting algorithm, in-place element-wise operations must not change the shape of the in-place array as a result of broadcasting. From 94bd0cb306f59e69189b09241b3db2350392d598 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Mon, 10 Jan 2022 11:38:15 -0500 Subject: [PATCH 4/6] Fix block quotes --- spec/API_specification/broadcasting.rst | 65 ++++++++++++------------- 1 file changed, 31 insertions(+), 34 deletions(-) diff --git a/spec/API_specification/broadcasting.rst b/spec/API_specification/broadcasting.rst index b930aaa12..db14a41fe 100644 --- a/spec/API_specification/broadcasting.rst +++ b/spec/API_specification/broadcasting.rst @@ -71,48 +71,45 @@ Examples The following examples demonstrate the application of the broadcasting algorithm for two compatible arrays. :: - A (4d array): 8 x 1 x 6 x 1 - B (3d array): 7 x 1 x 5 - --------------------------------- - Result (4d array): 8 x 7 x 6 x 5 - A (2d array): 5 x 4 - B (1d array): 1 - ------------------------- - Result (2d array): 5 x 4 - - A (2d array): 5 x 4 - B (1d array): 4 - ------------------------- - Result (2d array): 5 x 4 - - A (3d array): 15 x 3 x 5 - B (3d array): 15 x 1 x 5 - ------------------------------ - Result (3d array): 15 x 3 x 5 - - A (3d array): 15 x 3 x 5 - B (2d array): 3 x 5 - ------------------------------ - Result (3d array): 15 x 3 x 5 - - A (3d array): 15 x 3 x 5 - B (2d array): 3 x 1 - ------------------------------ - Result (3d array): 15 x 3 x 5 + A (4d array): 8 x 1 x 6 x 1 + B (3d array): 7 x 1 x 5 + --------------------------------- + Result (4d array): 8 x 7 x 6 x 5 + A (2d array): 5 x 4 + B (1d array): 1 + ------------------------- + Result (2d array): 5 x 4 + A (2d array): 5 x 4 + B (1d array): 4 + ------------------------- + Result (2d array): 5 x 4 + A (3d array): 15 x 3 x 5 + B (3d array): 15 x 1 x 5 + ------------------------------ + Result (3d array): 15 x 3 x 5 + A (3d array): 15 x 3 x 5 + B (2d array): 3 x 5 + ------------------------------ + Result (3d array): 15 x 3 x 5 + A (3d array): 15 x 3 x 5 + B (2d array): 3 x 1 + ------------------------------ + Result (3d array): 15 x 3 x 5 The following examples demonstrate array shapes which do **not** broadcast. :: - A (1d array): 3 - B (1d array): 4 # dimension does not match - A (2d array): 2 x 1 - B (3d array): 8 x 4 x 3 # second dimension does not match + A (1d array): 3 + B (1d array): 4 # dimension does not match + + A (2d array): 2 x 1 + B (3d array): 8 x 4 x 3 # second dimension does not match - A (3d array): 15 x 3 x 5 - B (2d array): 15 x 3 # singleton dimensions can only be prepended, not appended + A (3d array): 15 x 3 x 5 + B (2d array): 15 x 3 # singleton dimensions can only be prepended, not appended In-place Semantics ------------------ From 7597b740e0b41c889e0c2074eaee39ec812e57c5 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Mon, 17 Jan 2022 12:01:01 -0500 Subject: [PATCH 5/6] Fix rendering --- spec/API_specification/broadcasting.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/broadcasting.rst b/spec/API_specification/broadcasting.rst index 93a4519c6..58a49c768 100644 --- a/spec/API_specification/broadcasting.rst +++ b/spec/API_specification/broadcasting.rst @@ -49,15 +49,15 @@ The results of the element-wise operation must be stored in an array having a sh #. If ``d1 == 1``, then - - set the ``i``th element of ``shape`` to ``d2``. + - set the ``i``\th element of ``shape`` to ``d2``. #. Else, if ``d2 == 1``, then - - set the ``i``th element of ``shape`` to ``d1``. + - set the ``i``\th element of ``shape`` to ``d1``. #. Else, if ``d1 == d2``, then - - set the ``i``th element of ``shape`` to ``d1``. + - set the ``i``\th element of ``shape`` to ``d1``. #. Else, throw an exception. From fbddbd087c62d3947d2a00f91ff69441833a9ab7 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Tue, 18 Jan 2022 12:10:15 -0500 Subject: [PATCH 6/6] Fix nested list --- spec/API_specification/broadcasting.rst | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/spec/API_specification/broadcasting.rst b/spec/API_specification/broadcasting.rst index 58a49c768..ec72fb089 100644 --- a/spec/API_specification/broadcasting.rst +++ b/spec/API_specification/broadcasting.rst @@ -39,29 +39,27 @@ The results of the element-wise operation must be stored in an array having a sh #. Repeat, while ``i >= 0`` - #. Let ``n1`` be ``N1 - N + i``. + #. Let ``n1`` be ``N1 - N + i``. - #. If ``n1 >= 0``, let ``d1`` be the size of dimension ``n1`` for array ``A`` (i.e., the result of ``shape1[n1]``); else, let ``d1`` be ``1``. + #. If ``n1 >= 0``, let ``d1`` be the size of dimension ``n1`` for array ``A`` (i.e., the result of ``shape1[n1]``); else, let ``d1`` be ``1``. - #. Let ``n2`` be ``N2 - N + i``. + #. Let ``n2`` be ``N2 - N + i``. - #. If ``n2 >= 0``, let ``d2`` be the size of dimension ``n2`` for array ``B`` (i.e., the result of ``shape2[n2]``); else, let ``d2`` be ``1``. + #. If ``n2 >= 0``, let ``d2`` be the size of dimension ``n2`` for array ``B`` (i.e., the result of ``shape2[n2]``); else, let ``d2`` be ``1``. - #. If ``d1 == 1``, then + #. If ``d1 == 1``, then set the ``i``\th element of ``shape`` to ``d2``. - - set the ``i``\th element of ``shape`` to ``d2``. + #. Else, if ``d2 == 1``, then - #. Else, if ``d2 == 1``, then + - set the ``i``\th element of ``shape`` to ``d1``. - - set the ``i``\th element of ``shape`` to ``d1``. + #. Else, if ``d1 == d2``, then - #. Else, if ``d1 == d2``, then + - set the ``i``\th element of ``shape`` to ``d1``. - - set the ``i``\th element of ``shape`` to ``d1``. + #. Else, throw an exception. - #. Else, throw an exception. - - #. Set ``i`` to ``i-1``. + #. Set ``i`` to ``i-1``. #. Let ``tuple(shape)`` be the shape of the result array.