You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/create-deployment.mdx
+6-5Lines changed: 6 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -17,8 +17,9 @@ dates:
17
17
-[Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
18
18
19
19
1. Click the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard.
20
-
2. Click **Deploy a model** to launch the model deployment wizard.
21
-
3. Provide the necessary information:
20
+
2. From the drop-down menu, select the geographical region where you want to create your deployment.
21
+
3. Click **Deploy a model** to launch the model deployment wizard.
22
+
4. Provide the necessary information:
22
23
- Select the desired model and quantization to use for your deployment [from the available options](/managed-inference/reference-content/).
23
24
<Messagetype="important">
24
25
Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
@@ -28,12 +29,12 @@ dates:
28
29
</Message>
29
30
- Choose the geographical **region** for the deployment.
30
31
- Specify the GPU Instance type to be used with your deployment.
31
-
4. Enter a **name** for the deployment, and optional tags.
32
-
5. Configure the **network connectivity** settings for the deployment:
32
+
5. Enter a **name** for the deployment, and optional tags.
33
+
6. Configure the **network connectivity** settings for the deployment:
33
34
- Attach to a **Private Network** for secure communication and restricted availability. Choose an existing Private Network from the drop-down list, or create a new one.
34
35
- Set up **Public connectivity** to access resources via the public internet. Authentication by API key is enabled by default.
35
36
<Messagetype="important">
36
37
- Enabling both private and public connectivity will result in two distinct endpoints (public and private) for your deployment.
37
38
- Deployments must have at least one endpoint, either public or private.
38
39
</Message>
39
-
6. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments.
40
+
7. Click **Deploy model** to launch the deployment process. Once the model is ready, it will be listed among your deployments.
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/delete-deployment.mdx
+5-4Lines changed: 5 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -22,10 +22,11 @@ Once you have finished your inference tasks you can delete your deployment. This
22
22
-[Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
23
23
24
24
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
25
-
2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Iconname="more" /> to access the deployment dashboard.
26
-
3. Click the **Settings** tab of your deployment to display additional settings.
27
-
4. Click **Delete deployment**.
28
-
5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
25
+
2. From the drop-down menu, select the geographical region you want to manage.
26
+
3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Iconname="more" /> to access the deployment dashboard.
27
+
4. Click the **Settings** tab of your deployment to display additional settings.
28
+
5. Click **Delete deployment**.
29
+
6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
29
30
30
31
<Messagetype="important">
31
32
Deleting a deployment is a permanent action and will erase all its associated data.
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/import-custom-model.mdx
+9-8Lines changed: 9 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -24,25 +24,26 @@ Scaleway provides a selection of common models for deployment from the Scaleway
24
24
-[Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) to perform actions in your Organization.
25
25
26
26
1. Click **Managed Inference** in the **AI** section of the side menu in the [Scaleway console](https://console.scaleway.com/) to access the dashboard.
27
-
2. Click **Deploy a model** to launch the model deployment wizard.
28
-
3. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard.
29
-
4. Choose an upload source:
27
+
2. From the drop-down menu, select the geographical region you want to manage.
28
+
3. Click **Deploy a model** to launch the model deployment wizard.
29
+
4. In the **Choose a model** section, select **Custom model**. If you have no model yet, click **Import a model** to start the model import wizard.
30
+
5. Choose an upload source:
30
31
-**Hugging Face**: Pull the model from Hugging Face.
31
32
-**Object Storage**: This feature is coming soon.
32
-
5. Enter your Hugging Face access token, which must have READ access to the repository.
33
+
6. Enter your Hugging Face access token, which must have READ access to the repository.
33
34
<Messagetype="note">
34
35
[Learn how to generate a Hugging Face access token](https://huggingface.co/docs/hub/security-tokens).
35
36
</Message>
36
-
6. Enter the name of the Hugging Face repository to pull the model from.
37
+
7. Enter the name of the Hugging Face repository to pull the model from.
37
38
<Messagetype="note">
38
39
Ensure you have access to gated models if applicable. Refer to the [Hugging Face documentation](https://huggingface.co/docs/hub/en/models-gated) for details.
39
40
</Message>
40
-
7. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later.
41
-
8. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility.
41
+
8. Choose a name for your model. The name must be unique within your Organization and Project and cannot be changed later.
42
+
9. Click **Verify import** to check your Hugging Face credentials and ensure model compatibility.
42
43
<Messagetype="tip">
43
44
For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
44
45
</Message>
45
-
9. Review the summary of your import, which includes:
46
+
10. Review the summary of your import, which includes:
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/manage-allowed-ips.mdx
+5-4Lines changed: 5 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -28,13 +28,14 @@ Allowed IPs restrict the IPs allowed to access your Managed Inference endpoints.
28
28
## How to allow an IP address to connect to a deployment
29
29
30
30
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
31
-
2. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
32
-
3. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays.
33
-
4. Click **Add allowed IP**. The IP can be a single IP or an IP block.
31
+
2. From the drop-down menu, select the geographical region you want to manage.
32
+
3. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
33
+
4. Click the **Security** tab and navigate to the **Allowed IPs** section. A list of your allowed IP addresses displays.
34
+
5. Click **Add allowed IP**. The IP can be a single IP or an IP block.
34
35
<Messagetype="note">
35
36
The IP must be specified in CIDR format, i.e. `198.51.100.135/32` for a single IP or `198.51.100.0/24` for an IP block.
36
37
</Message>
37
-
5. Enter a single IP address or a subnetwork.
38
+
6. Enter a single IP address or a subnetwork.
38
39
<Messagetype="note">
39
40
To restore initial settings and allow connections from all IPs, delete all allowed IPs from the list.
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/managed-inference-with-private-network.mdx
+10-8Lines changed: 10 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -29,25 +29,27 @@ Using a Private Network for communications between your Instances hosting your a
29
29
### Attaching a Private Network during deployment setup
30
30
31
31
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
32
-
2. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays.
33
-
3. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section.
34
-
4. You will be asked to **attach a Private Network**. Two options are available:
32
+
2. From the drop-down menu, select the geographical region you want to manage.
33
+
3. Navigate to the **Deployments** section and click **Create New Deployment**. The setup wizard displays.
34
+
4. During the [setup process](/managed-inference/how-to/create-deployment/), you access the **Networking** section.
35
+
5. You will be asked to **attach a Private Network**. Two options are available:
35
36
-**Attach an existing Private Network**: Select from the list of available networks.
36
37
-**Add a new Private Network**: Choose this option if you need to create a new network.
37
-
5.**Confirm your selection** and complete the deployment setup process.
38
+
6.**Confirm your selection** and complete the deployment setup process.
38
39
39
40
### Attaching a Private Network to an existing deployment
40
41
41
42
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
42
-
2. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
43
-
3. Go to the **Overview** tab and locate the **Endpoints** section.
44
-
4. Click **Attach Private Network**. Two options are available:
43
+
2. From the drop-down menu, select the geographical region you want to manage.
44
+
3. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
45
+
4. Go to the **Overview** tab and locate the **Endpoints** section.
46
+
5. Click **Attach Private Network**. Two options are available:
45
47
-**Attach an existing Private Network**: Select from the list of available networks.
46
48
-**Add a new Private Network**: Choose this option if you need to create a new network.
47
49
<Messagetype="tip">
48
50
Alternatively, you can access the **Security tab** and attach a network from the **Private Network** section.
49
51
</Message>
50
-
5.**Save your changes** to apply the new network configuration.
52
+
6.**Save your changes** to apply the new network configuration.
Copy file name to clipboardExpand all lines: pages/managed-inference/how-to/monitor-deployment.mdx
+6-5Lines changed: 6 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -24,8 +24,9 @@ This documentation page shows you how to monitor your Managed Inference deployme
24
24
## How to monitor your LLM dashboard
25
25
26
26
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
27
-
2. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
28
-
3. Click the **Monitoring** tab of your deployment. The Cockpit overview displays.
29
-
4. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface.
30
-
5. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays.
31
-
6. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics.
27
+
2. From the drop-down menu, select the geographical region you want to manage.
28
+
3. Click a deployment name or <Iconname="more" /> > **More info** to access the deployment dashboard.
29
+
4. Click the **Monitoring** tab of your deployment. The Cockpit overview displays.
30
+
5. Click **Open Grafana metrics dashboard** to open your Cockpit's Grafana interface.
31
+
6. Authenticate with your [Grafana credentials](/cockpit/how-to/retrieve-grafana-credentials/). The Grafana dashboard displays.
32
+
7. Select your Managed Inference dashboard from the [list of your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/) to visualize your metrics.
Copy file name to clipboardExpand all lines: pages/managed-inference/quickstart.mdx
+18-14Lines changed: 18 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -32,8 +32,9 @@ Here are some of the key features of Scaleway Managed Inference:
32
32
## How to create a Managed Inference deployment
33
33
34
34
1. Navigate to the **AI** section of the [Scaleway console](https://console.scaleway.com/), and select **Managed Inference** from the side menu to access the Managed Inference dashboard.
35
-
2. Click **Create deployment** to launch the deployment creation wizard.
36
-
3. Provide the necessary information:
35
+
2. From the drop-down menu, select the geographical region where you want to create your deployment.
36
+
3. Click **Create deployment** to launch the deployment creation wizard.
37
+
4. Provide the necessary information:
37
38
- Select the desired model and the quantization to use for your deployment [from the available options](/managed-inference/reference-content/).
38
39
<Messagetype="important">
39
40
Scaleway Managed Inference allows you to deploy various AI models, either from the Scaleway catalog or by importing a custom model. For detailed information about supported models, visit our [Supported models in Managed Inference](/managed-inference/reference-content/supported-models/) documentation.
@@ -43,24 +44,25 @@ Here are some of the key features of Scaleway Managed Inference:
43
44
</Message>
44
45
- Choose the geographical **region** for the deployment.
45
46
- Specify the GPU Instance type to be used with your deployment.
46
-
4. Enter a **name** for the deployment, along with optional tags to aid in organization.
47
-
5. Configure the **network** settings for the deployment:
47
+
5. Enter a **name** for the deployment, along with optional tags to aid in organization.
48
+
6. Configure the **network** settings for the deployment:
48
49
- Enable **Private Network** for secure communication and restricted availability within Private Networks. Choose an existing Private Network from the drop-down list, or create a new one.
49
50
- Enable **Public Network** to access resources via the public Internet. API key protection is enabled by default.
50
51
<Messagetype="important">
51
52
- Enabling both private and public networks will result in two distinct endpoints (public and private) for your deployment.
52
53
- Deployments must have at least one endpoint, either public or private.
53
54
</Message>
54
-
6. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments.
55
+
7. Click **Create deployment** to launch the deployment process. Once the deployment is ready, it will be listed among your deployments.
55
56
56
57
## How to access a Managed Inference deployment
57
58
58
59
Managed Inference deployments have authentication enabled by default. As such, your endpoints expect a secret key generated with Scaleway's Identity and Access Management service (IAM) for authentication.
59
60
60
61
1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays.
61
-
2. Click <Iconname="more" /> next to the deployment you want to edit. The deployment dashboard displays.
62
-
3. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays.
63
-
4. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**.
62
+
2. From the drop-down menu, select the geographical region where you want to manage.
63
+
3. Click <Iconname="more" /> next to the deployment you want to edit. The deployment dashboard displays.
64
+
4. Click **Generate key** in the **Deployment connection** section of the dashboard. The token creation wizard displays.
65
+
5. Fill in the [required information for API key creation](/iam/how-to/create-api-keys/) and click **Generate API key**.
64
66
65
67
<Messagetype="tip">
66
68
You have full control over authentication from the **Security** tab of your deployment. Authentication is enabled by default.
@@ -69,8 +71,9 @@ Managed Inference deployments have authentication enabled by default. As such, y
69
71
## How to interact with Managed Inference
70
72
71
73
1. Click **Managed Inference** in the **AI** section of the side menu. The Managed Inference dashboard displays.
72
-
2. Click <Iconname="more" /> next to the deployment you want to edit. The deployment dashboard displays.
73
-
3. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal.
74
+
2. From the drop-down menu, select the geographical region where you want to manage.
75
+
3. Click <Iconname="more" /> next to the deployment you want to edit. The deployment dashboard displays.
76
+
4. Click the **Inference** tab. Code examples in various environments display. Copy and paste them into your code editor or terminal.
74
77
75
78
<Messagetype="note">
76
79
Prompt structure may vary from one model to another. Refer to the specific instructions for use in our [dedicated documentation](/managed-inference/reference-content/).
@@ -79,10 +82,11 @@ Managed Inference deployments have authentication enabled by default. As such, y
79
82
## How to delete a deployment
80
83
81
84
1. Click **Managed Inference** in the **AI** section of the [Scaleway console](https://console.scaleway.com) side menu. A list of your deployments displays.
82
-
2. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Iconname="more" /> to access the deployment dashboard.
83
-
3. Click the **Settings** tab of your deployment to display additional settings.
84
-
4. Click **Delete deployment**.
85
-
5. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
85
+
2. From the drop-down menu, select the geographical region where you want to create your deployment.
86
+
3. Choose a deployment either by clicking its name or selecting **More info** from the drop-down menu represented by the icon <Iconname="more" /> to access the deployment dashboard.
87
+
4. Click the **Settings** tab of your deployment to display additional settings.
88
+
5. Click **Delete deployment**.
89
+
6. Type **DELETE** to confirm and click **Delete deployment** to delete your deployment.
86
90
87
91
<Messagetype="important">
88
92
Deleting a deployment is a permanent action, and will erase all its associated configuration and resources.
0 commit comments