Skip to content

Update data interchange page with notes on DLPack #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 9, 2021
Merged

Update data interchange page with notes on DLPack #325

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions spec/design_topics/data_interchange.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,11 @@ The interchange mechanism must offer the following:
C/C++, and are released independently from each other. Hence a stable C
ABI is required for packages to work well together._

The best candidate for this protocol is DLPack. See the
[RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1)
for details.
The best candidate for this protocol is
[DLPack](https://github.com/dmlc/dlpack), and hence that is what this
standard has chosen as the primary/recommended protocol. Note that the
`asarray` function also supports the Python buffer protocol (CPU-only) to
support libraries that already implement buffer protocol support.

```{note}

Expand All @@ -64,10 +66,26 @@ hardly has any support from array libraries. CPU interoperability is
mostly dealt with via the NumPy-specific `__array__` (which, when called,
means the object it is attached to must return a `numpy.ndarray`
containing the data the object holds).

See the [RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1)
for discussion that preceded the adoption of DLPack.
```

## DLPack support

:::{note}
DLPack is a standalone protocol/project and can therefore be used outside of
this standard. Python libraries that want to implement only DLPack support
are recommended to do so using the same syntax and semantics as outlined
below. They are not required to return an array object from `from_dlpack`
which conforms to this standard.

DLPack itself has no documentation currently outside of the inline comments in
[dlpack.h](https://github.com/dmlc/dlpack/blob/main/include/dlpack/dlpack.h).
In the future, the below content may be migrated to the (to-be-written) DLPack docs.
:::

## Syntax for data interchange with DLPack
### Syntax for data interchange with DLPack

The array API will offer the following syntax for data interchange:

Expand All @@ -80,7 +98,7 @@ The array API will offer the following syntax for data interchange:
stream, e.g. in the case of multiple GPUs) and to access the data.


## Semantics
### Semantics

DLPack describe the memory layout of strided, n-dimensional arrays.
When a user calls `y = from_dlpack(x)`, the library implementing `x` (the
Expand All @@ -106,7 +124,7 @@ In the common case of the default stream being used, synchronization will be
unnecessary so asynchronous execution is enabled.


## Implementation
### Implementation

_Note that while this API standard largely tries to avoid discussing implementation details, some discussion and requirements are needed here because data interchange requires coordination between implementers on, e.g., memory management._

Expand Down