Scale test results for 1.1 (nginx#1346)

Adding scale test results and comparison summary for v1.1

Author: miledxz

tests/scale/results/1.1.0/1.1.0.md

@@ -0,0 +1,224 @@
+# Results for v1.1.0
+
+<!-- TOC -->
+- [Results for v1.1.0](#results-for-v110)
+  - [Summary](#summary)
+  - [Versions](#versions)
+  - [Tests](#tests)
+    - [Scale Listeners](#scale-listeners)
+    - [Scale HTTPS Listeners](#scale-https-listeners)
+    - [Scale HTTPRoutes](#scale-httproutes)
+    - [Scale Upstream Servers](#scale-upstream-servers)
+    - [Scale HTTP Matches](#scale-http-matches)
+  - [Future Improvements](#future-improvements)
+<!-- TOC -->
+
+## Summary
+
+- Memory usage has increased since 1.0
+- Number of events being processed has increased a bit, therefore so has the average time.
+- CPU and TTR seem to have stayed about the same.
+- One of the CPU charts for the final test was not being calculated properly.
+
+## Versions
+
+NGF version:
+
+```text
+commit: 7a37efac52546f2fe24dbc2772e913a49329ed6c
+date: 2023-12-06T02:17:06Z
+version: edge
+```
+
+with NGINX:
+
+```text
+nginx/1.25.3
+built by gcc 12.2.1 20220924 (Alpine 12.2.1_git20220924-r10)
+OS: Linux 5.15.109+
+```
+
+Kubernetes:
+
+```json
+  "serverVersion": {
+    "major": "1",
+    "minor": "27",
+    "gitVersion": "v1.27.3-gke.100",
+    "gitCommit": "6466b51b762a5c49ae3fb6c2c7233ffe1c96e48c",
+    "gitTreeState": "clean",
+    "buildDate": "2023-06-23T09:27:28Z",
+    "goVersion": "go1.20.5 X:boringcrypto",
+    "compiler": "gc",
+    "platform": "linux/amd64"
+  }
+```
+
+## Tests
+
+### Scale Listeners
+
+Reloads:
+
+| Total | Total Errors | Ave Time (ms)      | <= 500ms |
+|-------|--------------|--------------------|----------|
+| 128   | 0            | 136.38250641586066 | 100%     |
+
+
+Event Batch Processing:
+
+| Total | Ave Time (ms)      | <= 500ms | <= 1000ms |
+|-------|--------------------|----------|-----------|
+| 467   | 228.32680399999995 | 89.07%   | 97.43%    |
+
+**NGINX Errors**: None.
+
+**NGF Errors**: None.
+
+**Pod Restarts**: None.
+
+**CPU**: ![CPU.png](/tests/scale/results/1.1.0/TestScale_Listeners/CPU.png).
+
+**Memory**: ![Memory.png](/tests/scale/results/1.1.0/TestScale_Listeners/Memory.png).
+
+**Time To Ready**: ![TTR.png](/tests/scale/results/1.1.0/TestScale_Listeners/TTR.png).
+
+### Scale HTTPS Listeners
+
+Reloads:
+
+| Total | Total Errors | Ave Time (ms)      | <= 500ms |
+|-------|--------------|--------------------|----------|
+| 106   | 0            | 153.87735849056605 | 100%     |
+
+
+Event Batch Processing:
+
+| Total | Ave Time (ms)      | <= 500ms | <= 1000ms | <= 5000ms |
+|-------|--------------------|----------|-----------|-----------|
+| 520   | 266.9285714285714  | 92.5%    | 100%      | 100%      |
+
+
+**NGINX Errors**: None.
+
+**NGF Errors**: None.
+
+**Pod Restarts**: None.
+
+**CPU**: ![CPU.png](/tests/scale/results/1.1.0/TestScale_HTTPSListeners/CPU.png).
+
+**Memory**: ![Memory.png](/tests/scale/results/1.1.0/TestScale_HTTPSListeners/Memory.png).
+
+**Time To Ready**: ![TTR.png](/tests/scale/results/1.1.0/TestScale_HTTPSListeners/TTR.png).
+
+### Scale HTTPRoutes
+
+Reloads:
+
+| Delay     | Total | Total Errors | Ave Time (ms)     | <= 500ms | <= 1000ms |
+|-----------|-------|--------------|-------------------|----------|-----------|
+| 2 seconds | 1001  | 0            | 364.405076142132  | 75.92%   | 100%      |
+| No delay  | 1001  | 0            | 358.1523316062176 | 75.92%   | 100%      |
+
+
+Event Batch Processing:
+
+| Delay     | Total | Ave Time           | <= 500ms | <= 1000ms |
+|-----------|-------|--------------------|----------|-----------|
+| 2 seconds | 2056  | 200.38335809806838 | 83.37%   | 100%      |
+| No delay  | 2044  | 197.871000507872   | 83.22%   | 100%      |
+
+
+**NGINX Errors**: None.
+
+**NGF Errors**: None.
+
+**Pod Restarts**: None.
+
+**CPU**:
+
+2-sec delay:
+![CPU.png](/tests/scale/results/1.1.0/TestScale_HTTPRoutes/CPU.png).
+
+No delay:
+![CPU.png](/tests/scale/results/1.1.0/TestScale_HTTPRoutes/CPU-no-delay.png).
+
+**Memory**:
+
+2-sec delay:
+![Memory.png](/tests/scale/results/1.1.0/TestScale_HTTPRoutes/Memory.png).
+
+No delay:
+![Memory.png](/tests/scale/results/1.1.0/TestScale_HTTPRoutes/Memory-no-delay.png).
+
+**TTR**:
+
+Combined:
+![TTR.png](/tests/scale/results/1.1.0/TestScale_HTTPRoutes/TTR.png)
+
+### Scale Upstream Servers
+
+| Start Time (UNIX) | End Time (UNIX) | Duration (s) |
+|-------------------|-----------------|--------------|
+| 1701893886        | 1701893955      | 69           |
+
+Reloads:
+
+| Total | Total Errors | Ave Time (ms)       | <= 500ms |
+|-------|--------------|---------------------|----------|
+| 157   | 0            | 1126.36305732484075 | 100%     |
+
+Event Batch Processing:
+
+| Total | Ave Time (ms)      | <=500ms | <= 1000ms | <= 5000ms |
+|-------|--------------------|---------|-----------|-----------|
+| 159   | 210.08176100628933 | 99.37%  | 100%      | 100%      |
+
+**NGINX Errors**: None.
+
+**NGF Errors**: None.
+
+**Pod Restarts**: None.
+
+**CPU**: ![CPU.png](/tests/scale/results/1.1.0/TestScale_UpstreamServers/CPU.png).
+
+**Memory**: ![Memory.png](/tests/scale/results/1.1.0/TestScale_UpstreamServers/Memory.png).
+
+### Scale HTTP Matches
+
+**Results for the first match**:
+
+```text
+Running 30s test @ http://cafe.example.com
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    25.09ms   57.91ms 693.45ms   97.96%
+    Req/Sec   284.70     24.62   330.00     84.30%
+  16710 requests in 30.01s, 5.91MB read
+Requests/sec:    556.84
+Transfer/sec:    201.74KB
+```
+
+**Results for the last match**:
+
+```text
+Running 30s test @ http://cafe.example.com
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    17.63ms    1.93ms  56.53ms   86.77%
+    Req/Sec   284.44     18.22   323.00     74.29%
+  17025 requests in 30.10s, 6.02MB read
+Requests/sec:    565.66
+Transfer/sec:    204.94KB
+```
+
+**Findings**:
+
+- There's not a noticeable difference between the response times for the first match and last match. In
+fact, the latency of the last match is slightly lower than the latency of the first match.
+
+## Future Improvements
+
+- Check that the statuses of the Gateway API resources are updated after each scaling event.
+- Measure the time it takes for NGF to update the status of the Gateway API resources after creating or updating the resources.
+- Record the reload time distributions per bucket after each test.