diff --git a/pages/instances/troubleshooting/cant-connect-to-instance-with-pn-gateway.mdx b/pages/instances/troubleshooting/cant-connect-to-instance-with-pn-gateway.mdx index 42859c5bf0..820b54bfe4 100644 --- a/pages/instances/troubleshooting/cant-connect-to-instance-with-pn-gateway.mdx +++ b/pages/instances/troubleshooting/cant-connect-to-instance-with-pn-gateway.mdx @@ -22,4 +22,4 @@ The action to take depends on whether DHCP is [activated](/vpc/reference-content If it is not, disconnect the Instance from the Private Network, as there may be other factors impacting your Instance, like one of your Instances running a DHCP server. -If it is, this is expected behavior as all the traffic towards your Instance now goes through the Public Gateway. To access your Instance using SSH, first create a static NAT association between a port of your Public Gateway (eg 2222) and the private IP assigned to your Instance, on the SSH port (22 by default). Then, SSH to the Public Gateway's IP on port 2222. \ No newline at end of file +If it is, this is expected behavior as all the traffic towards your Instance now goes through the Public Gateway. The recommended solution to connect to your Instance is to use [SSH bastion](/public-gateways/how-to/use-ssh-bastion/#how-to-connect-to-a-resource-behind-your-ssh-bastion). \ No newline at end of file diff --git a/pages/vpc/reference-content/dns.mdx b/pages/vpc/reference-content/dns.mdx index 960fe4cd06..2a46471492 100644 --- a/pages/vpc/reference-content/dns.mdx +++ b/pages/vpc/reference-content/dns.mdx @@ -80,7 +80,7 @@ If you experience problems with DNS, try the following steps: 3. **Check Private Network name**. Issues can arise with Private Networks who share a name with a TLD. See our [dedicated document](/vpc/troubleshooting/pn-name/) for more help. -4. **Check whether you are using Network Manager**: DNS does not work out of the box for Linux distributions using Network Managed, such as **RockyLinux**. Find out how to resolve this problem [on our troubleshooting page](/vpc/troubleshooting/private-dns-dhcp-not-working/#distributions-running-network-manager) +4. **Check whether you are using Network Manager**: DNS does not work out of the box for Linux distributions using Network Managed, such as **RockyLinux**. Find out how to resolve this problem [on our troubleshooting page](/vpc/troubleshooting/pn-name/#distributions-running-network-manager) If you are still having problems reaching a resource attached to a Private Network via its hostname, [open a support ticket](https://console.scaleway.com/support). diff --git a/pages/vpc/troubleshooting/autoconfig-not-working.mdx b/pages/vpc/troubleshooting/autoconfig-not-working.mdx index 4b13d5b0ac..c73c13e772 100644 --- a/pages/vpc/troubleshooting/autoconfig-not-working.mdx +++ b/pages/vpc/troubleshooting/autoconfig-not-working.mdx @@ -1,9 +1,9 @@ --- meta: - title: My Instance is attached to a Private Network but auto-configuration is not working + title: My legacy Instance is attached to a Private Network but auto-configuration is not working description: Troubleshoot VPC autoconfiguration issues with this guide. Find solutions to common problems and ensure smooth network setup on Scaleway. content: - h1: My Instance is attached to a Private Network but auto-configuration is not working + h1: My legacy Instance is attached to a Private Network but auto-configuration is not working paragraph: Troubleshoot VPC autoconfiguration issues with this guide. Find solutions to common problems and ensure smooth network setup on Scaleway. tags: resource instance private-network auto-configuration auto configuration dhcp default-route dates: @@ -21,7 +21,7 @@ They leverage helper scripts provided by the `scaleway-ecosystem` package. These - When Instances are attached to Private Networks plugged into Public Gateways, make the default route received by DHCP the primary route for all traffic on the Instance - Keep the route to the Scaleway Metadata API more specific (see below). -If your Instance does not get auto-configured, it may be that you are using an old version of the `scaleway-ecosystem` package. `scaleway-ecosystem` version 0.0.4 or later is required. Use the following command to update it: +If your Instance does not get auto-configured, it may be that you are using an old Instance that has an old version of the `scaleway-ecosystem` package. `scaleway-ecosystem` version 0.0.4 or later is required. Use the following command to update it: - On Ubuntu (Focal and Jammy) or Debian (Stretch and Buster): diff --git a/pages/vpc/troubleshooting/connectivity-tests.mdx b/pages/vpc/troubleshooting/connectivity-tests.mdx new file mode 100644 index 0000000000..9adb20d729 --- /dev/null +++ b/pages/vpc/troubleshooting/connectivity-tests.mdx @@ -0,0 +1,125 @@ +--- +meta: + title: Connectivity tests for unreachable Instances in a Private Network + description: When your Scaleway Instance is unreachable via its attached Private Network, this page sets out a detailed testing process to help you diagnose and resolve the issue. +content: + h1: Connectivity tests for unreachable Instances in a Private Network + paragraph: When your Scaleway Instance is unreachable via its attached Private Network, this page sets out a detailed testing process to help you diagnose and resolve the issue. +tags: resource private-network connectivity ping test +dates: + validation: 2025-05-30 + posted: 2025-05-30 +categories: + - network +--- + +When an Instance becomes unreachable over a Private Network, it can be challenging to diagnose and resolve the issue. + +To help streamline the troubleshooting process, the Scaleway Network team has developed a step-by-step technique that can help identify and potentially fix connectivity problems. + +This technique involves a series of tests and checks to determine the root cause of the issue, including DNS resolution, ARP table verification, and Link-Local Address (LLA) calculation. + +By following these steps, you can quickly determine if the issue is related to DNS, network configuration, or resource-specific problems, and take corrective action to restore connectivity. + +It is particularly useful to carry out these steps before opening a support ticket, and supplying the results of the tests. This enables a swifter and more efficient resolution of your problem. + +## Step-by-step connectivity testing process + +### Step 1: Ping the non-communicating Instance from another Instance in the Private Network + +1. [Connect to another Instance](/instances/how-to/connect-to-instance/) in the same Private Network as the non-communicating Instance. + +2. Try to reach the non-communicating Instance with a ping via `ping instance-name` (where `instance-name` is the name you gave your Instance upon creation.) + +3. Carry out the same test for both IPv4 and IPv6, ie `ping -4 instance-name` and `ping -6 instance-name`. + + - If these tests fail, there may be a DNS issue meaning the Instance's hostname cannot be resolved, see step 4 directly below. + - Otherwise, if the tests succeed, continue to the section [Check ARP](#step-2-check-arp) + +4. Consider checking DNS records via the Scaleway Domains and DNS CLI as documented [here](/vpc/reference-content/dns/#troubleshooting), to see if the private DNS zone is correctly configured and the Instance's DNS records are up-to-date. + + + Note that we do not recommend editing these records yourself, rather open a support ticket with the information you have gathered. + + +### Step 2: Check ARP + +A server's **A**ddress **R**esolution **P**rotocol (ARP) table maps IP addresses to MAC addresses on a local network. Use this data to find the probe state for the target Instance's MAC address. + +1. From another Instance in the same Private Network as the non-communicating Instance, view ARP information via an `ip neigh` command. + + You should see an output of many lines in the following format: + + ``` + 172.16.8.2 dev ens5 lladdr 02:00:00:1c:0d:9b REACHABLE + ``` + +2. Find the line beginning with the non-communicating Instance's private IP address, and containing also its MAC address. You can check both via the Scaleway console in the Private Network's **Attached resources** tab. You can use the following command to identify the line, replacing `` with the Instance's MAC address: + + ``` + ip neigh | grep --ignore-case + ``` + + The line ends with the state of the probe for that Instance's MAC address. + + `FAILED` shows that the system tried to resolve the MAC address but did not get a response. + + Continue to the next stage of testing. + +### Step 3: Try reaching the Instance using its LLA + +If the Instance could not be reached via the resolution of its MAC address, next you should try to reach it via its IPv6 **L**ink **L**ocal **A**ddress (LLA) in the Private Network. + +The LLA is calculated from the Instance's MAC address. + +1. Use an online tool or guide to carry out the standardized process of calculating the Instance's LLA from its MAC address. Remember that you can view its MAC address from the Scaleway console, in the Private Network's **Attached Resources** tab. + + + You can use the following code snippet to calculate the LLA: + + ``` + bash-5.2$ mac_to_ipv6_ll() { + IFS=':'; set $1; unset IFS + echo "fe80::$(printf %02x $((0x$1 ^ 2)))$2:${3}ff:fe$4:$5$6" + } + ``` +
+ Example usage and output for a MAC address of `02:00:00:23:e4:01`: + + ``` + bash-5.2$ mac_to_ipv6_ll 02:00:00:23:e4:01 + fe80::0000:00ff:fe23:e401 + ``` + + +2. Ping the LLA. Ensure that you specify which interface to ping over (the Private Network) by adding the `%interface` zone index to the end of the ping command. For example, for a LLA of `fe80::0:0ff:fe1c:d9b` and a Private Network interface name of `ens5`, use the following command: + + ``` + ping -6 fe80::0:0ff:fe1c:d9b%ens5 + ``` + + + The interface name depends on the Instance type and OS. You can use the following command to get a compact summary of all network interfaces on your Instance: + + ``` + ip -brief -color addr show + ``` + + + A successful ping will show results similar to the following: + + ``` + PING fe80::0:0ff:fe1c:d9b%ens5 (fe80::ff:fe1c:d9b%ens5) 56 data bytes + 64 bytes from fe80::ff:fe1c:d9b%ens5: icmp_seq=1 ttl=64 time=1.27 ms + 64 bytes from fe80::ff:fe1c:d9b%ens5: icmp_seq=2 ttl=64 time=0.652 ms + 64 bytes from fe80::ff:fe1c:d9b%ens5: icmp_seq=3 ttl=64 time=0.632 ms + --- fe80::0:0ff:fe1c:d9b%ens5 ping statistics --- + 3 packets transmitted, 3 received, 0% packet loss, time 2042ms + rtt min/avg/max/mdev = 0.632/0.850/1.266/0.294 ms + ``` + + This response suggests the Instance is attached to the Private Network, but has lost its IPAM-provided IPv4 and IPv6 addresses. There could be a configuration issue on the Instance, such as DHCP client issue, a problem with `iptables` rules, or another parameter you may have manually altered. + + You can connect to your Instance via SSH using the LLA, to take control and attempt to fix the issue. + + If you do not believe you have modified any parameters affecting the Instance's DHCP or IP configuration over the Private Network, open a support ticket providing the full output from all of the above tests. diff --git a/pages/vpc/troubleshooting/index.mdx b/pages/vpc/troubleshooting/index.mdx index bb3b2a6229..4d85fff5ec 100644 --- a/pages/vpc/troubleshooting/index.mdx +++ b/pages/vpc/troubleshooting/index.mdx @@ -64,11 +64,11 @@ categories: ## VPC troubleshooting pages - - - - - - - - - - + - + - - + - + - + - - diff --git a/pages/vpc/troubleshooting/pn-name.mdx b/pages/vpc/troubleshooting/pn-name.mdx index 5c21cb9863..159dae838f 100644 --- a/pages/vpc/troubleshooting/pn-name.mdx +++ b/pages/vpc/troubleshooting/pn-name.mdx @@ -1,26 +1,36 @@ --- meta: - title: I am experiencing SSL or DNS errors and conflicts with my Private Network - description: This page helps you troubleshoot problems that can occur when you have given your Private Network a name that is also a TLD, which can cause errors and conflicts with SSL, DNS and addressing. + title: I am experiencing DNS or SSL errors and conflicts with my Private Network + description: This page helps you troubleshoot problems that can occur when accessing your resource by its hostname over a Private Network content: - h1: I am experiencing SSL or DNS errors and conflicts with my Private Network - paragraph: This page helps you troubleshoot problems that can occur when you have given your Private Network a name that is also a TLD, which can cause errors and conflicts with SSL, DNS and addressing. + h1: I am experiencing DNS or SSL errors and conflicts with my Private Network + paragraph: This page helps you troubleshoot problems that can occur when accessing your resource by its hostname over a Private Network tags: private-network tld ssl ssl-validation-failed dns name addressing dates: - validation: 2025-02-13 + validation: 2025-05-30 posted: 2024-01-31 categories: - network --- -## TLD naming problem with Private Networks +Scaleway VPC and Private Networks integrate managed DNS, for the effective resolution of hostnames to IP addresses. You may be experiencing DNS and/or related SSL errors when attempting to access your resource over its [hostname](/vpc/reference-content/dns/#hostname-format) on a Private Network. + +This document guides you through solutions to this type of problem. + +## Basic steps to try first + +- Make sure that [DHCP is activated](/vpc/how-to/activate-dhcp/) on your Private Network. +- Ensure that you have tried [detaching/reattaching](/vpc/troubleshooting/resource-attached-no-ip/#no-ip-address-is-displayed-but-i-have-activated-dhcp) the affected resources from/to the Private Network, particularly if they are older resources or you have [changed their name](/vpc/how-to/attach-resources-to-pn/#how-to-access-a-resource-on-a-private-network-via-its-hostname-dns). +- Ensure that you are correctly addressing resources attached to a Private Network, using the format `resource-name.private-network-name.internal`. The `.internal` is important to allow Scaleway DNS to distinguish public and private hostnames and correctly resolve requests. [Find out more about DNS](/vpc/reference-content/dns/). + +## Check for TLD naming problem If you have given your Private Network a name that is also a **T**op **L**evel **D**omain, you may experience SSL or addressing errors and conflicts when you try to access resources on the public internet with the same TLD as your Private Network, or when trying to reach resources on your Private Network from the public internet. Examples are: - Errors when trying to reach `google.dev` from a Private Network named `dev` - Errors when trying to reach `scw.cloud.network` from a Private Network named `network` -## Solutions +### Solution Scaleway is working on a solution to allow users to name their Private Networks with a TLD name without experiencing any conflicts. In the meantime, until this solution is deployed, you should **not** give your Private Network a name that is also a TLD. @@ -38,4 +48,22 @@ You can change your Private Network name at any time in the [Scaleway console](h - After changing the name of Private Network, it can take up to five minutes for the new name to be propagated. - You should also renew DHCP leases in order to update the list of search domains for the Private Network: do this by running `netplan apply` on Ubuntu servers. - Note that resources in a Private Network are in any case always reachable by their IP address. - \ No newline at end of file + + +## Check whether resource has a dot in its hostname + +Issues may arise with internal addressing if you give your resource a name that includes a dot (`.`). + +### Solution + +Try renaming your resource to a name that does not include a dot, then detach/reattach it to the Private Network in order to update DNS. + +## Distributions running Network Manager + +Private DNS should work out of the box for attached resources with Debian-based distributions, and for distributions using `systemd-resolved`, such as Ubuntu. + +It does **not** work out of the box for Linux distributions using Network Manager, such as **RockyLinux**. + +### Solution + +If your resource is running RockyLinux, or another distribution using Network Manager, you should manually configure the system to use the nameserver `169.254.169.254`. You will then be able to resolve the name of your resources, as well as public domain names. \ No newline at end of file diff --git a/pages/vpc/troubleshooting/private-dns-dhcp-not-working.mdx b/pages/vpc/troubleshooting/private-dns-dhcp-not-working.mdx index c4d52141bd..7d17db46d8 100644 --- a/pages/vpc/troubleshooting/private-dns-dhcp-not-working.mdx +++ b/pages/vpc/troubleshooting/private-dns-dhcp-not-working.mdx @@ -5,7 +5,7 @@ meta: content: h1: Private DNS and/or DHCP are not working paragraph: This page helps you troubleshoot problems when Private DNS and/or DHCP are not working -tags: resource instance private-network vpc private-DNS DNS DHCP debian network-manager linux rockylinux nameserver archlinux classeless-static-route systemd-networkd +tags: resource instance private-network vpc private-DNS DNS dates: validation: 2025-02-04 posted: 2023-07-13 @@ -13,32 +13,6 @@ categories: - network --- -There are some cases when a Private Network's private DNS or DHCP may not work 'out of the box' for attached resources. Read on to find out how to fix these problems. - -## Basic steps to try first - -- Make sure that [DHCP is activated](/vpc/how-to/activate-dhcp/) on your Private Network. -- Ensure that you have tried [detaching/reattaching](/vpc/troubleshooting/resource-attached-no-ip/#no-ip-address-is-displayed-but-i-have-activated-dhcp) the affected resources from/to the Private Network, particularly if they are older resources or you have [changed their name](/vpc/how-to/attach-resources-to-pn/#how-to-access-a-resource-on-a-private-network-via-its-hostname-dns). -- Ensure that you are correctly addressing resources attached to a Private Network, using the format `resource-name.private-network-name.internal`. The `.internal` is important to allow Scaleway DNS to distinguish public and private hostnames and correctly resolve requests. [Find out more about DNS](/vpc/reference-content/dns/). - -## Resources with a dot in their hostname - -Issues may arise with internal addressing if you give your resource a name that includes a dot (`.`). Try renaming your resource to a name that does not include a dot, then detach/reattach it to the Private Network in order to update DNS. - -## Private Networks with a name that is also a TLD - -See our [dedicated document](/vpc/troubleshooting/pn-name/) for a full explanation of this problem, and suggested solutions. - -## Distributions running Network Manager - -Private DNS should work out of the box for attached resources with Debian-based distributions, and for distributions using `systemd-resolved`, such as Ubuntu. - -It does **not** work out of the box for Linux distributions using Network Manager, such as **RockyLinux**. If your resource is running RockyLinux, or another distribution using Network Manager, you should manually configure the system to use the nameserver `169.254.169.254`. You will then be able to resolve the name of your resources, as well as public domain names. - -## DHCP is not working - -DHCP should work out of the box for attached resources running any and all Linux distributions. However, when it comes to **ArchLinux**, we are aware of a problem when it comes to setting a Classless Static Route to our service IP address `169.254.169.254`. - -In this case, if you are using `systemd-networkd` and do not have a route to our service IP address `169.254.169.254`, you should try updating and upgrading the packages on your system. - -There is also a known [bug](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867625) affecting **Debian Bullseye**, that prevents DHCP clients from correctly applying the default route. For this specific bug, we recommend using another distribution, such as **Debian Bookworm**. \ No newline at end of file +See: +- [I am experiencing DNS or SSL errors and conflicts with my Private Network](/vpc/troubleshooting/pn-name/) +- [My resource is attached to a Private Network but has no IP address on the network](/vpc/troubleshooting/resource-attached-no-ip/) diff --git a/pages/vpc/troubleshooting/resource-attached-no-ip.mdx b/pages/vpc/troubleshooting/resource-attached-no-ip.mdx index 1ab9ee355e..f2db0410d6 100644 --- a/pages/vpc/troubleshooting/resource-attached-no-ip.mdx +++ b/pages/vpc/troubleshooting/resource-attached-no-ip.mdx @@ -18,6 +18,7 @@ When you attach a resource to a Private Network, the Private Network's built-in - [Find out how to attach a resource to a Private Network](/vpc/how-to/attach-resources-to-pn/#how-to-attach-a-resource-to-a-private-network) - [Find out how to view your resource's IP address on the Private Network](/vpc/how-to/attach-resources-to-pn/#how-to-view-the-resources-ip-address) - [Find out how to specify the IP address to use when attaching a resource](/ipam/how-to/reserve-ip/) +- [Find out how to run connectivity tests to diagnose connectivity problems](/vpc/troubleshooting/connectivity-tests/) IP addresses are allocated asynchronously from the attachment of the resource in most cases. The IP will therefore not be immediately returned upon attaching a resource via the API, and there may be a brief delay before it displays in the console, as shown below: @@ -45,4 +46,12 @@ To fix this problem, **detach your resource from the Private Network, and reatta Note that some manual configuration of the network interface is required for Elastic Metal servers. Follow the steps in our [dedicated documentation](/elastic-metal/how-to/use-private-networks/#how-to-configure-the-network-interface-on-your-elastic-metal-server-for-private-networks). -You can also attach custom resources, such as virtual machines hosted on your Elastic Metal server, to Private Networks, by specifying their MAC addresses upon attachment/ Follow the instructions for attaching such a resource in the [Private Networks documentation](/vpc/how-to/attach-resources-to-pn/). \ No newline at end of file +You can also attach custom resources, such as virtual machines hosted on your Elastic Metal server, to Private Networks, by specifying their MAC addresses upon attachment/ Follow the instructions for attaching such a resource in the [Private Networks documentation](/vpc/how-to/attach-resources-to-pn/). + +### ArchLinux and Debian Bullseye + +DHCP should work out of the box for attached resources running any and all Linux distributions. However, when it comes to **ArchLinux**, we are aware of a problem when it comes to setting a Classless Static Route to our service IP address `169.254.169.254`. + +In this case, if you are using `systemd-networkd` and do not have a route to our service IP address `169.254.169.254`, you should try updating and upgrading the packages on your system. + +There is also a known [bug](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867625) affecting **Debian Bullseye**, that prevents DHCP clients from correctly applying the default route. For this specific bug, we recommend using another distribution, such as **Debian Bookworm**. \ No newline at end of file diff --git a/pages/vpc/troubleshooting/vpc-pn-routing-connectivity-issues.mdx b/pages/vpc/troubleshooting/vpc-pn-routing-connectivity-issues.mdx index 575ca8e929..4e58727335 100644 --- a/pages/vpc/troubleshooting/vpc-pn-routing-connectivity-issues.mdx +++ b/pages/vpc/troubleshooting/vpc-pn-routing-connectivity-issues.mdx @@ -49,11 +49,13 @@ Using the same MAC address on VMs in different AZs, or switching such MAC addres ## My resources cannot communicate via their hostnames -See our dedicated documentation on [resolving private DNS errors](/vpc/troubleshooting/private-dns-dhcp-not-working/). +See our dedicated documentation on [resolving private DNS errors](/vpc/troubleshooting/pn-name/). ## I am experiencing other connectivity issues Check the [Scaleway Status page](https://status.scaleway.com/), to see whether there are any ongoing incidents which could affect the connectivity or network access of your resources. +Carry out our [recommended tests](/vpc/troubleshooting/connectivity-tests/) for thorough connectivity troubleshooting and resolution. +