From 7267050378d717574ee14bd1ea5adb5e42af066c Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Mon, 13 Sep 2021 02:34:45 -0400 Subject: [PATCH 1/8] add to_device --- spec/API_specification/array_object.md | 30 ++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index 7d9640715..c44cc4006 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -1191,3 +1191,33 @@ Evaluates `self_i ^ other_i` for each element of an array instance with the resp Element-wise results must equal the results returned by the equivalent element-wise function [`bitwise_xor(x1, x2)`](elementwise_functions.md#bitwise_xorx1-x2-). ``` + +(method-to_device)= +### to_device(self, device, /, *, stream=None) + +Copy the array from the device it currently resides to the specified `device`. + +#### Parameters + +- **self**: _<array>_ + + - array instance. + +- **device**: _<device>_ + + - device to place the copied array on. + +- **stream**: _Optional\[ Union\[ int, Any ]]_ + + - stream object to use during copy. See {ref}`method-__dlpack__`. + +#### Returns + +- **out**: _<array>_ + + - an identical copy of the source array placed on the target `device`. + +```{note} + +Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization (which is out of scope of this standard) is required to guarantee data safety, the library should explain to its users. +``` From 09ea66890f01306fa5a908c7995b3b279ed85467 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Mon, 13 Sep 2021 02:51:18 -0400 Subject: [PATCH 2/8] clarify device object --- spec/design_topics/device_support.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/spec/design_topics/device_support.md b/spec/design_topics/device_support.md index 7788e2949..cd13d0f9d 100644 --- a/spec/design_topics/device_support.md +++ b/spec/design_topics/device_support.md @@ -49,9 +49,16 @@ cross-device data transfer: a `Device` object, to move an array to a different device. ```{note} -The only way to obtain a `Device` object is from the `.device` property on -the array object, hence there is no `Device` object in the array API itself -that can be instantiated to point to a specific physical or logical device. +In the current API standard, the only way to obtain a `Device` object is from the +`.device` property on the array object, hence there is no `Device` object in the +array API itself that can be instantiated to point to a specific physical or +logical device. In other words, the standard does *not* include a universal +`Device` object recognized by all compliant libraries. + +For array libraries that concern with multi-device support, including CPU and GPU, +it is free to expose a library-specific device object for use (ex: creating an +array on a particular device). For the purpose of this standard, it is considered +an (important) implementation detail. ``` From 7fd3f209dca4e604750384edf8ebc74b5fc4e460 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Fri, 8 Oct 2021 17:25:46 -0400 Subject: [PATCH 3/8] clarify library-specific stream can be used --- spec/API_specification/array_object.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index f8a1f3ad7..385aba188 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -1229,7 +1229,7 @@ Copy the array from the device it currently resides to the specified `device`. - **stream**: _Optional\[ Union\[ int, Any ]]_ - - stream object to use during copy. See {ref}`method-__dlpack__`. + - stream object to use during copy. In addition to the supported types as discussed in {ref}`method-__dlpack__`, any library-specific stream object is also allowed to be used here. #### Returns @@ -1239,6 +1239,5 @@ Copy the array from the device it currently resides to the specified `device`. ```{note} -Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization (which is out of scope of this standard) is required to guarantee data safety, the library should explain to its users. +If `stream` is given, the copy operation will be enqueued on it; otherwise, it is enqueued on the default stream/queue (the concept of which is out of scope of this standard). Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization (which is also out of scope of this standard) is required to guarantee data safety, the library should explain to its users. ``` - From 829b5a9df71b60b1aa66e2f6398b1c98031a6de7 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Fri, 8 Oct 2021 17:28:28 -0400 Subject: [PATCH 4/8] move -> copy --- spec/design_topics/device_support.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/design_topics/device_support.md b/spec/design_topics/device_support.md index 650b490d9..044e69be4 100644 --- a/spec/design_topics/device_support.md +++ b/spec/design_topics/device_support.md @@ -48,8 +48,8 @@ cross-device data transfer: libraries is out of scope). 2. A `device=None` keyword for array creation functions, which takes an instance of a `Device` object. -3. A `.to_device(device)` method on the array object, with `device` again being - a `Device` object, to move an array to a different device. +3. A `.to_device` method on the array object, with `device` again being + a `Device` object, to copy an array to a different device. ```{note} In the current API standard, the only way to obtain a `Device` object is from the From d4b7a14dcd084fc16cabd93bee42369ed7fd014d Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Sun, 17 Oct 2021 20:18:57 -0400 Subject: [PATCH 5/8] apply review comments --- spec/API_specification/array_object.md | 7 ++++--- spec/design_topics/device_support.md | 8 ++++---- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index 385aba188..0605c05d3 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -458,6 +458,8 @@ Exports the array for consumption by {ref}`function-from_dlpack` as a DLPack cap For other device types which do have a stream, queue or similar synchronization mechanism, the most appropriate type to use for `stream` is not yet determined. E.g., for SYCL one may want to use an object containing an in-order `cl::sycl::queue`. This is allowed when libraries agree on such a convention, and may be standardized in a future version of this API standard. + Support for any `stream` value other than `None` is optional and up to each library in question. + Device-specific notes: :::{admonition} CUDA @@ -1229,7 +1231,7 @@ Copy the array from the device it currently resides to the specified `device`. - **stream**: _Optional\[ Union\[ int, Any ]]_ - - stream object to use during copy. In addition to the supported types as discussed in {ref}`method-__dlpack__`, any library-specific stream object is also allowed to be used here. + - stream object to use during copy. In addition to the supported types as discussed in {ref}`method-__dlpack__`, any library-specific stream object is also allowed to be used here, with the caveat that using such an object would make the code non-portable. #### Returns @@ -1238,6 +1240,5 @@ Copy the array from the device it currently resides to the specified `device`. - an array with the same data and dtype, located on the specified `device`. ```{note} - -If `stream` is given, the copy operation will be enqueued on it; otherwise, it is enqueued on the default stream/queue (the concept of which is out of scope of this standard). Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization (which is also out of scope of this standard) is required to guarantee data safety, the library should explain to its users. +If `stream` is given, the copy operation will be enqueued on it; otherwise, it is enqueued on the default stream/queue. Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization is required to guarantee data safety, the library should explain to its users. ``` diff --git a/spec/design_topics/device_support.md b/spec/design_topics/device_support.md index 044e69be4..8a543ed0b 100644 --- a/spec/design_topics/device_support.md +++ b/spec/design_topics/device_support.md @@ -58,10 +58,10 @@ array API itself that can be instantiated to point to a specific physical or logical device. In other words, the standard does *not* include a universal `Device` object recognized by all compliant libraries. -For array libraries that concern with multi-device support, including CPU and GPU, -it is free to expose a library-specific device object for use (ex: creating an -array on a particular device). For the purpose of this standard, it is considered -an (important) implementation detail. +For array libraries which concern themselves with multi-device support, including CPU and GPU, +they are free to expose a library-specific device object for use (e.g., for creating an +array on a particular device). It can be used as an input to `to_device`; the caveat +is that it would make the code specific to that library. ``` From 7690ee62751493e546dcbe88c010e5f77f6b28b4 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Fri, 22 Oct 2021 02:15:44 -0400 Subject: [PATCH 6/8] Apply suggestions from code review Co-authored-by: Athan --- spec/API_specification/array_object.md | 8 ++++---- spec/design_topics/device_support.md | 18 +++++++++--------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index 0605c05d3..8d15fca19 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -1217,7 +1217,7 @@ Element-wise results must equal the results returned by the equivalent element-w (method-to_device)= ### to\_device(self, device, /, *, stream=None) -Copy the array from the device it currently resides to the specified `device`. +Copy the array from the device on which it currently resides to the specified `device`. #### Parameters @@ -1227,7 +1227,7 @@ Copy the array from the device it currently resides to the specified `device`. - **device**: _<device>_ - - a `device` object (see {ref}`device-support`) + - a `device` object (see {ref}`device-support`). - **stream**: _Optional\[ Union\[ int, Any ]]_ @@ -1237,8 +1237,8 @@ Copy the array from the device it currently resides to the specified `device`. - **out**: _<array>_ - - an array with the same data and dtype, located on the specified `device`. + - an array with the same data and data type as `self` and located on the specified `device`. ```{note} -If `stream` is given, the copy operation will be enqueued on it; otherwise, it is enqueued on the default stream/queue. Whether the copy is performed synchronously or asynchronously is up to the array library. As a result, if any synchronization is required to guarantee data safety, the library should explain to its users. +If `stream` is given, the copy operation should be enqueued on the provided `stream`; otherwise, the copy operation should be enqueued on the default stream/queue. Whether the copy is performed synchronously or asynchronously is library dependent. As a result, if synchronization is required to guarantee data safety, this should be clearly explained in a conforming library's documentation. ``` diff --git a/spec/design_topics/device_support.md b/spec/design_topics/device_support.md index 8a543ed0b..7e8f0ff4b 100644 --- a/spec/design_topics/device_support.md +++ b/spec/design_topics/device_support.md @@ -48,20 +48,20 @@ cross-device data transfer: libraries is out of scope). 2. A `device=None` keyword for array creation functions, which takes an instance of a `Device` object. -3. A `.to_device` method on the array object, with `device` again being - a `Device` object, to copy an array to a different device. +3. A `.to_device` method on the array object to copy an array to a different device. ```{note} In the current API standard, the only way to obtain a `Device` object is from the -`.device` property on the array object, hence there is no `Device` object in the -array API itself that can be instantiated to point to a specific physical or -logical device. In other words, the standard does *not* include a universal -`Device` object recognized by all compliant libraries. +`.device` property on the array object. The standard does **not** include a universal +`Device` object recognized by all compliant libraries. Accordingly, the standard does +not provide a means of instantiating a `Device` object to point to a specific physical or +logical device. For array libraries which concern themselves with multi-device support, including CPU and GPU, -they are free to expose a library-specific device object for use (e.g., for creating an -array on a particular device). It can be used as an input to `to_device`; the caveat -is that it would make the code specific to that library. +they are free to expose a library-specific device object (e.g., for creating an +array on a particular device). While a library-specific device object can be used as input to +`to_device`, beware that this will mean non-portability as code will be specific to +that library. ``` From d34077bd572d00a397ec1ea21d062bb05e8eb342 Mon Sep 17 00:00:00 2001 From: Athan Date: Mon, 1 Nov 2021 11:48:28 -0700 Subject: [PATCH 7/8] Update copy --- spec/API_specification/array_object.md | 9 +++++---- spec/design_topics/device_support.md | 2 ++ 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index 8d15fca19..cf90f8e84 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -458,8 +458,9 @@ Exports the array for consumption by {ref}`function-from_dlpack` as a DLPack cap For other device types which do have a stream, queue or similar synchronization mechanism, the most appropriate type to use for `stream` is not yet determined. E.g., for SYCL one may want to use an object containing an in-order `cl::sycl::queue`. This is allowed when libraries agree on such a convention, and may be standardized in a future version of this API standard. - Support for any `stream` value other than `None` is optional and up to each library in question. - + ```{note} + Support for a `stream` value other than `None` is optional and implementation-dependent. + ``` Device-specific notes: :::{admonition} CUDA @@ -1231,7 +1232,7 @@ Copy the array from the device on which it currently resides to the specified `d - **stream**: _Optional\[ Union\[ int, Any ]]_ - - stream object to use during copy. In addition to the supported types as discussed in {ref}`method-__dlpack__`, any library-specific stream object is also allowed to be used here, with the caveat that using such an object would make the code non-portable. + - stream object to use during copy. In addition to the types supported in {ref}`method-__dlpack__`, implementations may choose to support any library-specific stream object with the caveat that any code using such an object would not be portable. #### Returns @@ -1240,5 +1241,5 @@ Copy the array from the device on which it currently resides to the specified `d - an array with the same data and data type as `self` and located on the specified `device`. ```{note} -If `stream` is given, the copy operation should be enqueued on the provided `stream`; otherwise, the copy operation should be enqueued on the default stream/queue. Whether the copy is performed synchronously or asynchronously is library dependent. As a result, if synchronization is required to guarantee data safety, this should be clearly explained in a conforming library's documentation. +If `stream` is given, the copy operation should be enqueued on the provided `stream`; otherwise, the copy operation should be enqueued on the default stream/queue. Whether the copy is performed synchronously or asynchronously is implementation-dependent. Accordingly, if synchronization is required to guarantee data safety, this must be clearly explained in a conforming library's documentation. ``` diff --git a/spec/design_topics/device_support.md b/spec/design_topics/device_support.md index 7e8f0ff4b..454171731 100644 --- a/spec/design_topics/device_support.md +++ b/spec/design_topics/device_support.md @@ -57,6 +57,8 @@ In the current API standard, the only way to obtain a `Device` object is from th not provide a means of instantiating a `Device` object to point to a specific physical or logical device. +The choice to not include a standardized `Device` object may be revisited in a future revision of this standard. + For array libraries which concern themselves with multi-device support, including CPU and GPU, they are free to expose a library-specific device object (e.g., for creating an array on a particular device). While a library-specific device object can be used as input to From 71019a91f35e6e2cce52c35e4b9c4e373c7c7da7 Mon Sep 17 00:00:00 2001 From: Athan Date: Mon, 1 Nov 2021 11:49:05 -0700 Subject: [PATCH 8/8] Add linebreak. --- spec/API_specification/array_object.md | 1 + 1 file changed, 1 insertion(+) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index cf90f8e84..ff5b1b46b 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -461,6 +461,7 @@ Exports the array for consumption by {ref}`function-from_dlpack` as a DLPack cap ```{note} Support for a `stream` value other than `None` is optional and implementation-dependent. ``` + Device-specific notes: :::{admonition} CUDA