Skip to content

App SDK Migration (to depend on Holoscan SDK) #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
76bf7c5
Squashed commit of the following:
MMelQin Jul 19, 2023
c46d01d
Fix black complaints
MMelQin Jul 19, 2023
aa64ea1
Local Flake8 did not complain this one.
MMelQin Jul 19, 2023
2bde738
Quiet mypy complaint
MMelQin Jul 19, 2023
0de65b6
Updated the notebook and included packaging and running the MAP
MMelQin Jul 21, 2023
dc57a8d
Updated the notebook after resovling issues with MAP extract command
MMelQin Jul 22, 2023
d702d75
Correct typo
MMelQin Jul 22, 2023
28f80cb
Updates with Holoscan v0.6 release version, and remove not needed tests
MMelQin Aug 4, 2023
c44dfb0
Fix complaint
MMelQin Aug 4, 2023
888bc71
Fix Flake complaint that "check --autofix" did not catch
MMelQin Aug 4, 2023
984d77f
Updated the app and notebook using the seg inference operator.
MMelQin Aug 10, 2023
82248eb
Updated, renamed, and removed Jupyter notebooks
MMelQin Aug 11, 2023
09ec8cc
Add files needed for packaging, and move test input file to default i…
MMelQin Aug 12, 2023
8b20388
Updated/added files for the tutorials section in the user's guide
MMelQin Aug 16, 2023
763ee06
Updated the tutorial section of the doc
MMelQin Aug 16, 2023
48c69cd
Exposing more holoscan modules and updated doc
MMelQin Aug 18, 2023
24fb6bf
Import sorting complaint
MMelQin Aug 18, 2023
a87f831
Import fewer Holoscan SDK operators in own module
MMelQin Aug 18, 2023
dc7f6b5
Even less HS operator imported due to dependency issue
MMelQin Aug 18, 2023
a282126
Updated the user doc
MMelQin Aug 24, 2023
e7bc536
Remove the packager command line options required during dev testing
MMelQin Aug 24, 2023
de1b7d5
Ping holoscan~=0.6.0 and updated section on ARM64 MAP packaging
MMelQin Aug 26, 2023
6d60040
Added command line option support for ease of running python app dire…
MMelQin Aug 30, 2023
30fcfdd
Fix format checking complaints
MMelQin Aug 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/developing_with_sdk/core_concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ classDiagram
Graph "1" --> "many" Operator : contains
```

[Application](/modules/_autosummary/monai.deploy.core.Application) represents a workflow as a [Graph](/modules/_autosummary/monai.deploy.core.graphs.Graph) and the graph handles [Operator](/modules/_autosummary/monai.deploy.core.Operator)s which are computational tasks.
[Application](/modules/_autosummary/monai.deploy.core.Application) represents a workflow as a [Graph](/modules/graphs) and the graph handles [Operator](/modules/_autosummary/monai.deploy.core.Operator)s which are computational tasks.

To develop and deploy your MONAI App, you can follow the steps below (click a node to see the detail):

Expand Down
48 changes: 22 additions & 26 deletions docs/source/developing_with_sdk/creating_application_class.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ caption: |
from monai.deploy.core import Application, env, resource


@resource(cpu=1, gpu=1, memory="2Gi")
# pip_packages can be a string that is a path(str) to requirements.txt file or a list of packages.
@env(pip_packages=["scikit-image >= 0.17.2"])
class App(Application):
"""This is a very basic application.

Expand All @@ -37,41 +34,34 @@ class App(Application):
pass

if __name__ == "__main__":
App(do_run=True)
App().run()

```

### Decorators

The resource requirements (such as `cpu`, `memory`, and `gpu`) for the application can be specified by using [@resource](/modules/_autosummary/monai.deploy.core.resource) decorator. This information is used only when the packaged app (Docker image) is executed.

[@env](/modules/_autosummary/monai.deploy.core.env) accepts `pip_packages` parameter as a string that is a path to requirements.txt file or a list of packages to install. If `pip_packages` is specified, the definition will be aggregated with the package dependency list of other operators. The aggregated requirement definitions are stored as a "[requirements.txt](https://pip.pypa.io/en/stable/cli/pip_install/#example-requirements-file)" file and it would be installed in [packaging time](/developing_with_sdk/executing_packaged_app_locally).


### compose() method

In `compose()` method, operators are instantiated and connected through <a href="../modules/_autosummary/monai.deploy.core.Application.html#monai.deploy.core.Application.add_flow">self.add_flow()</a>.

> add_flow(source_op, destination_op, io_map=None)
> add_flow(source_op, destination_op, port_pairs )

`io_map` is a dictionary of mapping from the source operator's label to the destination operator's label(s) and its type is `Dict[str, str|Set[str]]`.
`port_pairs` is a sequence of string tuples mapping from the source operator's named output(s) to the destination operator's named input(s) and its type is `Set[Tuple[str, str]]`.

We can skip specifying `io_map` if both the number of `source_op`'s outputs and the number of `destination_op`'s inputs are one.
For example, if Operators named `task1` and `task2` has only one input and output (with the label `image`), `self.add_flow(task1, task2)` is same with `self.add_flow(task1, task2, {"image": "image"})` or `self.add_flow(task1, task2, {"image": {"image"}})`.
We can skip specifying `Set[Tuple[str, str]]` if both the number of `source_op`'s outputs and the number of `destination_op`'s inputs are one.
For example, if Operators named `task1` and `task2` has only one input and output (with the label `image`), `self.add_flow(task1, task2)` is same with `self.add_flow(task1, task2, {("image", "image")})` or `self.add_flow(task1, task2, {("image", "image")})`.

```python
def compose(self):
task1 = Task1()
task2 = Task2()

self.add_flow(task1, task2)
# self.add_flow(task1, task2, {"image": "image"})
# self.add_flow(task1, task2, {"image": {"image"}})
# self.add_flow(task1, task2, {("image", "image")})
# self.add_flow(task1, task2, {("image", "image")})
```

> add_operator(operator)

If an operator in the workflow graph is both a root node and a leaf node, you can execute <a href="../modules/_autosummary/monai.deploy.core.Application.html#monai.deploy.core.Application.add_flow">self.add_operator()</a> for adding the operator to the workflow graph of the application.
If an operator in the workflow graph is both a root node and a leaf node, you can execute <a href="../modules/_autosummary/monai.deploy.core.Application.html#monai.deploy.core.Application.add_operator">self.add_operator()</a> for adding the operator to the workflow graph of the application.

```python
def compose(self):
Expand All @@ -83,7 +73,7 @@ If an operator in the workflow graph is both a root node and a leaf node, you ca

```python
if __name__ == "__main__":
App(do_run=True)
App().run()
```

The above lines in `app.py` are needed to execute the application code by using `python` interpreter.
Expand All @@ -101,9 +91,15 @@ caption: |
from app import App

if __name__ == "__main__":
App(do_run=True)
App().run()
```

## Package Dependency and Resource Requirements

Unlike in previous versions where Python decorators are used to define the resource requirements (such as `cpu`, `memory`, and `gpu`) for the application, a YAML file is required to store such information with sections and attributes as defined in the [MONAI Application Package Specification](https://github.com/Project-MONAI/monai-deploy/blob/main/guidelines/monai-application-package.md). This file is only needed when the application is packaged into a MONAI Application Package container image. When the MAP is run, the executor is expected to parse the resource requirements embedded in the MAP to ensure they are met in the host system.

Similarly, instead of using Python decorators, package dependencies of the application and all of its operators need to be aggregated and stored as a "[requirements.txt](https://pip.pypa.io/en/stable/cli/pip_install/#example-requirements-file)" file, to be installed at [packaging time](/developing_with_sdk/packaging_app).

## Creating a Reusable Application

Like <a href="./creating_operator_classes.html#creating-a-reusable-operator">Operator class</a>, an Application class can be implemented in a way that the common Application class can be reusable.
Expand Down Expand Up @@ -174,12 +170,12 @@ The above workflow can be expressed like below
notifier = Notifier()
writer = Writer()

self.add_flow(reader1, processor1, {"image": {"image1", "image2"},
"metadata": "metadata"})
self.add_flow(reader2, processor2, {"roi": "roi"})
self.add_flow(processor1, processor2, {"image": "image"})
self.add_flow(processor1, writer, {"image": "image"})
self.add_flow(reader1, processor1, {("image", "image1"), ("image", "image2"),
("metadata", "metadata")})
self.add_flow(reader2, processor2, {("roi", "roi")})
self.add_flow(processor1, processor2, {("image", "image")})
self.add_flow(processor1, writer, {("image", "image")})
self.add_flow(processor2, notifier)
self.add_flow(processor2, processor3)
self.add_flow(processor3, writer, {"seg_image": "seg_image"})
self.add_flow(processor3, writer, {("seg_image", "seg_image")})
```
51 changes: 17 additions & 34 deletions docs/source/developing_with_sdk/creating_operator_classes.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,26 +10,24 @@ lineno-start: 1
caption: |
An Operator class definition example
---
from typing import Any, Dict
from pathlib import Path
from monai.deploy.core import (ExecutionContext, Image, InputContext,
Operator, OutputContext, OperatorSpec)

import monai.deploy.core as md # 'md' stands for MONAI Deploy (or can use 'core' instead)
from monai.deploy.core import (DataPath, ExecutionContext, Image, InputContext,
IOType, Operator, OutputContext)


@md.input("image", DataPath, IOType.DISK)
@md.input("mask", Image, IOType.IN_MEMORY)
@md.output("image", Image, IOType.IN_MEMORY)
@md.output("metadata", Dict[str, Any], IOType.IN_MEMORY)
@md.env(pip_packages=["scikit-image>=0.17.2"])
class MyOperator(Operator):
"""Sample Operator implementation."""

def setup(self, spec: OperatorSpec):
spec.input("image_path")
spec.output("image")
spec.output("metadata")

def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
from skimage import filters, io

# Get input image
image_path = op_input.get("image").path
image_path = Path(op_input.receive("image_path")) # omitting validation for brevity
if image_path.is_dir():
image_path = next(image_path.glob("*.*")) # take the first file
input_image = io.imread(image_path)
Expand All @@ -44,24 +42,13 @@ class MyOperator(Operator):
metadata = {"shape": input_image.shape, "dtype": input_image.dtype}

# Set output
op_output.set(Image(output_image), "image")
op_output.set(metadata, "metadata")
op_output.emit(Image(output_image), "image")
op_output.emit(metadata, "metadata")
```

### Decorators

The input and output properties of the operator are specified by using [@input](/modules/_autosummary/monai.deploy.core.input) and [@output](/modules/_autosummary/monai.deploy.core.output) decorators.
### setup() method

[@input](/modules/_autosummary/monai.deploy.core.input) and [@output](/modules/_autosummary/monai.deploy.core.output) decorators accept (**`<Label>`**, [`<Data Type>`](/modules/domain_objects), [`<Storage Type>`](/modules/_autosummary/monai.deploy.core.IOType)) as parameters.

If no `@input` or `@output` decorator is specified, the following properties are used by default:

```python
@md.input("", DataPath, IOType.DISK) # if no @input decorator is specified.
@md.output("", DataPath, IOType.DISK) # if no @output decorator is specified.
```

[@env](/modules/_autosummary/monai.deploy.core.env) accepts `pip_packages` parameter as a string that is a path to requirements.txt file or a list of packages to install. If `pip_packages` is specified, the definition will be aggregated with the package dependency list of other operators and the application. The aggregated requirement definitions are stored as a "[requirements.txt](https://pip.pypa.io/en/stable/cli/pip_install/#example-requirements-file)" file and it would be installed in [packaging time](/developing_with_sdk/executing_packaged_app_locally).
In prior releases, the input and output properties of the operator are specified by using `@input` and `@output` decorators, but starting with release v0.6, the **<a href="../modules/_autosummary/monai.deploy.core.Operator.html#monai.deploy.core.Operator.setup">setup()</a>** method is used.

### compute() method

Expand All @@ -73,15 +60,11 @@ Please check the description of **<a href="../modules/_autosummary/monai.deploy.
2. Application's input/output
3. [Model](/modules/_autosummary/monai.deploy.core.models.Model)'s name/path/predictor

Note that, if the operator is a leaf operator in the workflow graph and the operator output's `(<data type>, <storage type>) == (DataPath, DISK)`, you cannot call `op_output.set()` method.
Instead, you can use the destination path available by `op_output.get().path` to store output data and the
following logic is expected:
Note that, if the operator is a leaf operator in the workflow graph and the operator output is file(s) written to the file system, the path needs to be set using the operator's constructor or a named input for path object or string. The leaf operator can also have in-memory output(s) without requiring other operator(s) as receiver, if the output is configured correctly like in the [GaussionOperator in Simple Image Processing App](https://github.com/Project-MONAI/monai-deploy-app-sdk/blob/main/examples/apps/simple_imaging_app/gaussian_operator.py)

```python
output_folder = op_output.get().path # get the output folder path
output_path = output_folder / "final_output.png" # get the output file path
imsave(output_path, data_out) # save the output data
```
## Package dependencies

The dependencies of the operator need to be captured in a "[requirements.txt](https://pip.pypa.io/en/stable/cli/pip_install/#example-requirements-file)", instead of using the `@env` decorator as in earlier releases. The aggregated requirement definitions for an application are then store in a consolidated "[requirements.txt](https://pip.pypa.io/en/stable/cli/pip_install/#example-requirements-file)" file, to be installed at [packaging time](/developing_with_sdk/packaging_app).

## Creating a Reusable Operator

Expand Down
6 changes: 3 additions & 3 deletions docs/source/developing_with_sdk/deploying_and_hosting_map.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Deploying and Hosting MONAI App Package

The MONAI Application Package, MAP, built and packaged using MONAI Deploy App SDK, can be deployed in multiple ways, each with different levels of integration with a hosting platform.
- A MAP is an OCI compliant container, albeit requiring <a href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/overview.html"> Nvidia Container Toolkit </a> as it is based on the <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch">Nvidia PyTorch Python container image </a>. The hosting platforms, via their own tooling, can inspect the MAP metadata and ensure dependencies are satisfied on launching the MAP container, for example, an MAP can used in a Argo workflow.
- A MAP runs "natively" on a platform, where the platform specific adaptor or shim embedded in the MAP can understand the MAP and the platform APIs thus manages the life cycle of the application per platform requests. At the onset, the only native execution planned is on the <a href="https://github.com/Project-MONAI/monai-deploy/releases">MONAI Deploy</a>. The support for other platforms is on the horizon.
- As envisioned for the long term, standardization of deployment package specification will be needed, so that compliant application packages shall work on all compliant platforms. One of the initiatives in this area is the <a href="https://oam.dev/">Open Appication Model</a>, though the App SDK project is not yet actively investigating the integration.
- A MAP is an OCI compliant container, albeit requiring <a href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/overview.html"> Nvidia Container Toolkit </a> as it is by default based on <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/containers/holoscan">Nvidia container image with GPU support</a>. The hosting platforms, via their own tooling, can inspect the MAP metadata and ensure dependencies are satisfied on launching the MAP container, for example, an MAP can be used in a [Argo workflow](https://argoproj.github.io/argo-workflows/), in [Docker Compose](https://docs.docker.com/compose/), or simply run using [Docker](https://www.docker.com/).
- A MAP runs "natively" on a platform, where the platform specific adaptor or utilities embedded in the MAP can understand the MAP and the platform APIs thus manage the life cycle of the application per platform requests. At the onset, the only native execution planned is on the <a href="https://github.com/Project-MONAI/monai-deploy/releases">MONAI Deploy</a>. The support for other platforms is on the horizon.
- As envisioned for the long term, standardization of deployment package specification will be needed, so that compliant application packages shall work on all compliant platforms. One of the initiatives in this area is the <a href="https://oam.dev/">Open Application Model</a>, where an MAP can be used as the Application component.
6 changes: 5 additions & 1 deletion docs/source/developing_with_sdk/designing_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ Those domain object classes are controllable by the SDK so can be optimized furt

The full list of Domain Object classes are available [here](/modules/domain_objects).

:::{note}
**`The functionality of mapping DataPath to the input and output of root and leaf operator(s) is absent starting with Release V0.6 of this App SDK, due to the move to rely on Holoscan SDK. It is planned to be re-introduced at a later time. For the time being, the application's input and output folders are passed to the root and leaf operators' constructor, as needed.`**

Among those classes, [**DataPath**](/modules/_autosummary/monai.deploy.core.domain.DataPath) data type is special.

- If an operator in the workflow graph is a root node (a node with no incoming edges) and its input's `(<data type>, <storage type>) == (DataPath, DISK)`, the input path given by the user [during the execution](/developing_with_sdk/executing_app_locally) would be mapped into the input of the operator.
Expand All @@ -102,9 +105,10 @@ In `A linear workflow` example above, if the workflow is processing the image da
- **Output** (`output3`): a file path ([`DataPath`](/modules/_autosummary/monai.deploy.core.domain.DataPath), [`DISK`](/modules/_autosummary/monai.deploy.core.IOType))

Note that `input1` and `output3` are [DataPath](/modules/_autosummary/monai.deploy.core.domain.DataPath) type with [IOType.DISK](/modules/_autosummary/monai.deploy.core.IOType). Those paths are mapped into input and output paths given by the user during the execution.
:::

:::{note}
The above workflow graph is the same as a [Simple Image Processing App](/getting_started/tutorials/01_simple_app). Please look at the tutorial to see how such an application can be made with MONAI Deploy App SDK.
The above workflow graph is the same as a [Simple Image Processing App](/getting_started/tutorials/simple_app). Please look at the tutorial to see how such an application can be made with MONAI Deploy App SDK.
:::

## Complex Workflows
Expand Down
Loading