Skip to content
This repository was archived by the owner on Nov 19, 2024. It is now read-only.

Updated custom VCL documentation #6833

Merged
merged 24 commits into from
Mar 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0d4d0cd
Updated instructions for creating custom VCL for IP allowlist
meker12 Mar 10, 2020
681c9e6
Merge branch 'master' into mae-magecloud-4821-fastly-allowlist-updates
meker12 Mar 11, 2020
c069a97
Update src/cloud/cdn/fastly-vcl-badreferer.md
meker12 Mar 13, 2020
2abfc95
Update src/cloud/cdn/fastly-vcl-blocking.md
meker12 Mar 13, 2020
53e7c90
Update src/cloud/trouble/robots-sitemap.md
meker12 Mar 13, 2020
fcf93a3
Update src/cloud/trouble/robots-sitemap.md
meker12 Mar 13, 2020
8240ba9
Update src/cloud/trouble/robots-sitemap.md
meker12 Mar 13, 2020
e3b29a8
Update src/cloud/trouble/robots-sitemap.md
meker12 Mar 13, 2020
6e1c8b0
Update src/cloud/cdn/fastly-vcl-blocking.md
meker12 Mar 13, 2020
518e0d3
Update src/cloud/cdn/fastly-vcl-blocking.md
meker12 Mar 13, 2020
ab37bc8
Update src/cloud/cdn/cloud-vcl-custom-snippets.md
meker12 Mar 13, 2020
e37fa78
Update src/cloud/cdn/fastly-vcl-blocking.md
meker12 Mar 13, 2020
4944537
Fixed linting error
meker12 Mar 13, 2020
39264f8
Merge branch 'master' into mae-magecloud-4821-fastly-allowlist-updates
meker12 Mar 16, 2020
37c6084
Update src/cloud/trouble/robots-sitemap.md
meker12 Mar 16, 2020
73cc701
Apply suggestions from code review
meker12 Mar 17, 2020
bae0ff9
Apply suggestions from code review
meker12 Mar 17, 2020
c526fff
Merge branch 'mae-magecloud-4821-fastly-allowlist-updates' of github.…
meker12 Mar 17, 2020
571fc5e
Apply suggestions from code review
meker12 Mar 17, 2020
a7eb86a
Changed execute to run per review feedback
meker12 Mar 17, 2020
fded85f
Fixed instance of "escape special characters"
meker12 Mar 17, 2020
0204f8a
Fix broken link
meker12 Mar 18, 2020
10edee4
Update src/cloud/cdn/fastly-vcl-blocking.md
meker12 Mar 18, 2020
491af33
Merge branch 'master' into mae-magecloud-4821-fastly-allowlist-updates
meker12 Mar 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/_data/toc/cloud-guide.yml
Original file line number Diff line number Diff line change
Expand Up @@ -270,11 +270,11 @@ pages:
url: /cloud/cdn/fastly-vcl-badreferer.html
versionless: true

- label: Secure access to the Magento Admin UI
- label: Custom VCL for IP allow list
url: /cloud/cdn/fastly-vcl-whitelist.html
versionless: true

- label: Custom VCL for blocking
- label: Custom VCL for IP block list
url: /cloud/cdn/fastly-vcl-blocking.html
versionless: true

Expand Down
9 changes: 4 additions & 5 deletions src/cloud/cdn/cloud-vcl-custom-snippets.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ You can create and manage custom VCL snippets from the Magento Admin UI or by us

### Example VCL snippet code {#vcl-curl}

The following example shows the custom VCL snippet that filters traffic by client IP address in JSON format.
The following example shows the custom VCL snippet (JSON format) that filters traffic by client IP address:

```json
{
Expand All @@ -59,7 +59,7 @@ The following example shows the custom VCL snippet that filters traffic by clien
```

{: .bs-callout-warning}
In this example, the VCL code is formatted as a JSON payload that can be saved to a file and submitted in a Fastly API request. If you submit the VCL code snippet from the Magento Admin UI, or as a JSON string using the Fastly API, you must escape special characters to prevent validation errors. See the example in [Secure the Magento Admin UI]({{ site.baseurl }}/cloud/cdn/fastly-vcl-whitelist.html#vcl).
In this example, the VCL code is formatted as a JSON payload that can be saved to a file and submitted in a Fastly API request. When sending the snippet as JSON for an API request, you must use a backslash to escape special characters in the code to prevent JSON validation errors. See [Using dynamic VCL snippets](https://docs.fastly.com/vcl/vcl-snippets/) in the Fastly VCL documentation. If you submit the VCL snippet from the Magento Admin UI, you do not have to escape special characters.

The VCL logic in the `content` field performs the following actions:

Expand Down Expand Up @@ -90,8 +90,7 @@ The *Custom VCL snippets* view shows only the snippets added through the Magento

See the following examples that show how to create and manage custom VCL snippets from the Magento Admin UI:

- [Secure access to the Magento Admin UI]({{ site.baseurl }}/cloud/cdn/fastly-vcl-whitelist.html)
- [Set up redirects to WordPress using Fastly]({{ site.baseurl }}/cloud/cdn/fastly-vcl-wordpress.html)
- [Custom VCL for IP allowlist]({{ site.baseurl }}/cloud/cdn/fastly-vcl-whitelist.html)
- [Block referral spam]({{ site.baseurl }}/cloud/cdn/fastly-vcl-badreferer.html)

## Manage custom VCL snippets using the API
Expand Down Expand Up @@ -306,4 +305,4 @@ Use these commands to manage snippets that you added using the Fastly API. If yo
<!-- Link definitions -->

[Manage custom VCL snippets]: {{site.baseurl}}/common/images/cloud/cloud-fastly-edit-snippets.png
{:width="650px"}
{:width="650px"}
22 changes: 10 additions & 12 deletions src/cloud/cdn/fastly-vcl-badreferer.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ We recommend adding custom VCL configurations to a Staging environment where you

- Configure the {{ site.var.data.ece }} environment for Fastly services. See [Set up Fastly]({{ site.baseurl }}/cloud/cdn/configure-fastly.html).

- Get Admin credentials for your {{ site.data.var.ece }} environment.
- Admin credentials to access the Magento Admin UI for your {{ site.data.var.ece }} environment

- Review your site logs for fake referral URLs and make a list of domains to block.
- Review your site logs for fake referral URLs, and make a list of domains to block.

## Create a referrer block list

Expand Down Expand Up @@ -59,43 +59,41 @@ For more information about Edge Dictionaries, see [Creating and using Edge Dicti

## Create a custom VCL snippet to block referrer spam

The following custom VCL snippet code (JSON format) checks incoming requests and blocks requests from any referrer site included in the `referrer_blocklist` edge dictionary.
The following custom VCL snippet code (JSON format) shows the logic to check and block requests. The VCL snippet captures the host of a referrer website into a header, and then compares the host name to the list of URLs in the `referrer_blocklist` dictionary. If the host name matches, the request is blocked with a `403 Forbidden` error.

```json
{
"name": "block_bad_referrer",
"dynamic": "0",
"type": "recv",
"priority": "5",
"content": "set req.http.Referer-Host = regsub(req.http.Referer, \"^https?://?([^:/\\s]+).*$\", \"\\1\"); if (table.lookup(referrer_blocklist, req.http.Referer-Host)) { error 403 \"Forbidden\"; }"
"content": "set req.http.Referer-Host = regsub(req.http.Referer, \"^https?:\/\/?([^:\/s]+).*$\", \"\\1\"); if (table.lookup(referrer_blocklist, req.http.Referer-Host)) { error 403 \"Forbidden\"; }"
}
```

Review the example code and change values as needed:
Before creating a snippet based on this example, review the values to determine whether you need to make any changes:

- `name` — Name for the VCL snippet. For this example, we used `block_bad_referrer`.

- `dynamic` — Value 0 indicates a [regular snippet](https://docs.fastly.com/guides/vcl-snippets/using-regular-vcl-snippets) to upload to the versioned VCL for the Fastly configuration.

- `priority` — Determines when the VCL snippet runs. The priority is `5` to run this snippet code before any of the default Magento VCL snippets (`magentomodule_*`) assigned a priority of 50.
- `priority` — Determines when the VCL snippet runs. The priority is `5` to run this snippet code before any of the default Magento VCL snippets (`magentomodule_*`) assigned a priority of 50. You must set the priority for each custom snippet higher or lower than 50 depending on when you want your snippet to run. Snippets with lower priority numbers run first.

- `type` — Specifies a location to insert the snippet in the VCL version. In this example, the VCL snippet is a `recv` snippet. When the snippet is inserted into the VCL version, it is added to the `vcl_recv` subroutine, below the default Fastly VCL code and above any objects.

- `content` — The snippet of VCL code to run in one line, without line breaks.

In this example, the VCL code logic captures the host of a referrer website into a header, and then compares the host name to the list of URLs in the `referrer_blocklist` dictionary.
After reviewing and updating the code for your environment, use either of the following methods to add the custom VCL snippet to your Fastly service configuration:

If the host name matches, the request is blocked with a `403 Forbidden` error.
- [Add the custom VCL snippet from the Magento Admin](#add-the-custom-vcl-snippet). This method is recommended if you can access the Magento Admin UI. (Requires [Fastly version 1.2.58]({{site.baseurl}}/cloud/cdn/configure-fastly.html#upgrade) or later.)

See the [Fastly VCL reference](https://docs.fastly.com/vcl/reference/) for information about creating Fastly VCL code snippets.

Add the custom VCL snippet to your Fastly service configuration from the Magento Admin UI (requires Fastly module 1.2.58 or later). If you cannot access the Admin UI, save the JSON code example in a file and upload it using the Fastly API. See [Creating a VCL snippet using the Fastly API]({{ site.baseurl }}/cloud/cdn/cloud-vcl-custom-snippets.html(#manage-custom-vcl-snippets-using-the-api).
- Save the JSON code example to a file (for example, `allowlist.json`) and [upload it using the Fastly API]({{site.baseurl}}/cloud/cdn/cloud-vcl-custom-snippets.html#manage-custom-vcl-snippets-using-the-api). Use this method if you cannot access the Magento Admin UI.

## Add the custom VCL snippet

{% include cloud/admin-ui-login-step.md %}

1. Click **Stores** > **Settings** > **Configuration** > **Advanced** > **System**.
1. Click **Stores** > Settings > **Configuration** > **Advanced** > **System**.

1. Expand **Full Page Cache** > **Fastly Configuration** > **Custom VCL Snippets**.

Expand Down
60 changes: 49 additions & 11 deletions src/cloud/cdn/fastly-vcl-blocking.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ You can use the Fastly CDN module for Magento 2 to create an Edge ACL with a lis
**Prerequisites:**

- List of client IP addresses to block
- Account access and URL for the Magento Admin UI for the Staging or Production environment
- Admin credentials to access the Magento Admin UI for your {{ site.data.var.ece }} environment
- Fastly API credentials for Staging and Production environments

## Create Edge ACL for blocking client IPs {#edge-acl}
Expand All @@ -31,16 +31,16 @@ If you want to manage access for both Staging and Production sites, create the E
1. Enter IP address values in the list. Any client IPs added to this list will be blocked access from the site.
1. Optionally, select the **Negated** checkbox if needed.

You will reference the Edge ACL by name in your VCL snippet code.
You reference the Edge ACL by name in your VCL snippet code.

## Create blocklist.json {#vcl}
## Create the custom VCL for the block list {#vcl}

{:.bs-callout-info}
This example shows advanced users how to create custom VCL code snippet to configure blocking rules that can be uploaded to the Fastly service using the Fastly API. You can also configure a blocklist or allowlist from the {{ site.data.var.ee }} Admin UI. See [Blocking](https://github.com/fastly/fastly-magento2/blob/master/Documentation/Guides/BLOCKING.md) in the Fastly CDN for Magento 2 module documentation.
This example shows advanced users how to create a VCL code snippet to configure custom blocking rules that can be uploaded to the Fastly service. You can configure a block list or allow list based on country from the {{ site.data.var.ee }} Admin UI using the [Blocking](https://github.com/fastly/fastly-magento2/blob/master/Documentation/Guides/BLOCKING.md) feature available in the Fastly CDN for Magento 2 module.

After you define the Edge ACL, you can use it to create the VCL snippet to block access to the IP addresses specified in the ACL. You can use the same VCL snippet in both Staging and Production environments, but you must upload the snippet to each environment separately.

Create a `blocklist.json` file with the following VCL code in JSON format:
The following custom VCL snippet code (JSON format) shows the logic to block incoming requests with a client IP address that matches an address in the blocklist ACL.

```json
{
Expand All @@ -52,24 +52,62 @@ Create a `blocklist.json` file with the following VCL code in JSON format:
}
```

Review the following parameter values and update your code snippet if necessary:
Before creating a snippet based on this example, review the values to determine whether you need to make any changes:

- `name`: Name for the VCL snippet. For this example, we used the name `blocklist`.
- `priority`: Determines the VCL snippet call order. Set the priority to 5 to immediately run and check for blocked IP addresses. This priority runs before any of the uploaded and default Magento VCL snippets (`magentomodule_`) that have a priority of 50.

- `priority`: Determines when the VCL snippet runs. The priority is `5` to immediately run and check whether a Magento Admin UI requests are coming from an allowed IP address. The snippet runs before any of the default Magento VCL snippets (`magentomodule_*`) assigned a priority of 50. You must set the priority for each custom snippet higher or lower than 50 depending on when you want your snippet to run. Snippets with lower priority numbers run first.

- `type`: Specifies the type of VCL snippet that determines the location of the snippet in the generated VCL code. In this example, we use `recv`, which inserts the VCL code in the `vcl_recv` subroutine, below the boilerplate VCL and above any objects. See the [Fastly VCL snippet reference](https://docs.fastly.com/api/config#api-section-snippet) for the list of snippet types.

- `content`: The snippet of VCL code to run, which checks the client IP address. If the IP is in the Edge ACL, it is blocked from access with a `403 Forbidden` error for the entire website. All other client IP addresses are allowed access.

{:.bs-callout-info}
The default VCL snippets include a prepended name of `magentomodule_` with a priority of 50. **Do not use the `magentomodule_` name** for your custom VCL Snippets. You must also set the priority for each custom snippet higher or lower than 50 depending on when you want your snippet to run. Lower priority numbers execute first.
After reviewing and updating the code for your environment, use either of the following methods to add the custom VCL snippet to your Fastly service configuration:

- [Add the custom VCL snippet from the Magento Admin](#add-the-custom-vcl-snippet). This method is recommended if you can access the Magento Admin UI. (Requires [Fastly version 1.2.58]({{site.baseurl}}/cloud/cdn/configure-fastly.html#upgrade) or later.)

- Save the JSON code example to a file (for example, `blocklist.json`) and [upload it using the Fastly API]({{site.baseurl}}/cloud/cdn/cloud-vcl-custom-snippets.html#manage-custom-vcl-snippets-using-the-api). Use this method if you cannot access the Magento Admin UI.

## Add the custom VCL snippet

{% include cloud/admin-ui-login-step.md %}

1. Click **Stores** > Settings > **Configuration** > **Advanced** > **System**.

1. Expand **Full Page Cache** > **Fastly Configuration** > **Custom VCL Snippets**.

## Finish adding the VCL {#complete}
1. Click **Create Custom Snippet**.

After saving the VCL snippet, add the VCL snippet to the Fastly service configuration. See [Add VCL snippets using the Fastly API]({{ site.baseurl }}/cloud/cdn/cloud-vcl-custom-snippets.html#add-vcl-snippets-using-the-fastly-api).
1. Add the VCL snippet values:

- **Name** — `blocklist`

- **Type** — `recv`

- **Priority** — `5`

- Add the **VCL** snippet content:

```conf
if ( client.ip ~ blocklist) { error 403 "Forbidden"; }
```

1. Click **Create** to generate the VCL snippet file with the name pattern `type_priority_name.vcl`, for example `recv_5_blocklist.vcl`

1. After the page reloads, click **Upload VCL to Fastly** in the *Fastly Configuration* section to add the file to the Fastly service configuration.

1. After the uploads, refresh the cache according to the notification at the top of the page.

Fastly validates the updated version of the VCL code during the upload process. If the validation fails, edit the custom VCL snippet to fix the issue. Then, upload the VCL again.

## Additional VCL examples for blocking requests

The following examples show how to block requests using inline condition statements instead of an ACL list.

{: .bs-callout-warning}
In these examples, the VCL code is formatted as a JSON payload that can be saved to a file and submitted in a Fastly API request. You can submit the [VCL snippet from the Admin UI](#add-the-custom-vcl-snippet), or as a JSON string using the Fastly API. If you use the Fastly API with a JSON string, you must use a backslash to escape special characters to prevent validation errors.
See [Using dynamic VCL snippets](https://docs.fastly.com/vcl/vcl-snippets/) in the Fastly VCL documentation.

### VCL code sample: Block by country code

This example uses the two-character ISO 3166-1 country code for the country associated with the IP address.
Expand Down
Loading