diff --git a/tests/results/dp-perf/1.3.0/1.3.0.md b/tests/results/dp-perf/1.3.0/1.3.0.md new file mode 100755 index 0000000000..239b1a754a --- /dev/null +++ b/tests/results/dp-perf/1.3.0/1.3.0.md @@ -0,0 +1,84 @@ +# Results + +## Test environment + +NGINX Plus: false + +GKE Cluster: + +- Node count: 12 +- k8s version: v1.28.9-gke.1000000 +- vCPUs per node: 16 +- RAM per node: 65855088Ki +- Max pods per node: 110 +- Zone: us-east1-b +- Instance Type: n2d-standard-16 + +## Summary + +- Performance seems to have improved. + +## Test1: Running latte path based routing + +```text +Requests [total, rate, throughput] 30000, 1000.03, 1000.00 +Duration [total, attack, wait] 30s, 29.999s, 934.498µs +Latencies [min, mean, 50, 90, 95, 99, max] 594.289µs, 940.484µs, 894.291µs, 1.076ms, 1.155ms, 1.475ms, 26.622ms +Bytes In [total, mean] 4740000, 158.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:30000 +Error Set: +``` + +## Test2: Running coffee header based routing + +```text +Requests [total, rate, throughput] 30000, 1000.00, 999.97 +Duration [total, attack, wait] 30.001s, 30s, 885.05µs +Latencies [min, mean, 50, 90, 95, 99, max] 630.729µs, 925.423µs, 909.768µs, 1.061ms, 1.117ms, 1.264ms, 10.285ms +Bytes In [total, mean] 4770000, 159.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:30000 +Error Set: +``` + +## Test3: Running coffee query based routing + +```text +Requests [total, rate, throughput] 30000, 1000.04, 1000.01 +Duration [total, attack, wait] 30s, 29.999s, 867.406µs +Latencies [min, mean, 50, 90, 95, 99, max] 649.447µs, 939.602µs, 913.399µs, 1.068ms, 1.137ms, 1.32ms, 36.366ms +Bytes In [total, mean] 5010000, 167.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:30000 +Error Set: +``` + +## Test4: Running tea GET method based routing + +```text +Requests [total, rate, throughput] 30000, 1000.02, 1000.00 +Duration [total, attack, wait] 30s, 29.999s, 798.43µs +Latencies [min, mean, 50, 90, 95, 99, max] 639.884µs, 941.213µs, 920.839µs, 1.067ms, 1.117ms, 1.266ms, 16.095ms +Bytes In [total, mean] 4680000, 156.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:30000 +Error Set: +``` + +## Test5: Running tea POST method based routing + +```text +Requests [total, rate, throughput] 30000, 1000.00, 999.97 +Duration [total, attack, wait] 30.001s, 30s, 1.006ms +Latencies [min, mean, 50, 90, 95, 99, max] 638.741µs, 951.815µs, 936.228µs, 1.085ms, 1.138ms, 1.261ms, 9.666ms +Bytes In [total, mean] 4680000, 156.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:30000 +Error Set: +``` diff --git a/tests/results/ngf-upgrade/1.3.0/1.3.0.md b/tests/results/ngf-upgrade/1.3.0/1.3.0.md new file mode 100755 index 0000000000..4654b4afce --- /dev/null +++ b/tests/results/ngf-upgrade/1.3.0/1.3.0.md @@ -0,0 +1,49 @@ +# Results + +## Test environment + +NGINX Plus: false + +GKE Cluster: + +- Node count: 12 +- k8s version: v1.28.9-gke.1000000 +- vCPUs per node: 16 +- RAM per node: 65855088Ki +- Max pods per node: 110 +- Zone: us-east1-b +- Instance Type: n2d-standard-16 + +## Summary + +- Similar results as last time. + +## Test: Send http /coffee traffic + +```text +Requests [total, rate, throughput] 6000, 100.02, 100.01 +Duration [total, attack, wait] 59.992s, 59.99s, 1.099ms +Latencies [min, mean, 50, 90, 95, 99, max] 795.938µs, 1.153ms, 1.119ms, 1.308ms, 1.386ms, 1.658ms, 13.45ms +Bytes In [total, mean] 954000, 159.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:6000 +Error Set: +``` + +![http.png](http.png) + +## Test: Send https /tea traffic + +```text +Requests [total, rate, throughput] 6000, 100.02, 100.00 +Duration [total, attack, wait] 59.999s, 59.991s, 7.907ms +Latencies [min, mean, 50, 90, 95, 99, max] 836.052µs, 1.21ms, 1.16ms, 1.38ms, 1.481ms, 1.842ms, 13.741ms +Bytes In [total, mean] 912000, 152.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:6000 +Error Set: +``` + +![https.png](https.png) diff --git a/tests/results/ngf-upgrade/1.3.0/http.png b/tests/results/ngf-upgrade/1.3.0/http.png new file mode 100644 index 0000000000..c738f8fd6b Binary files /dev/null and b/tests/results/ngf-upgrade/1.3.0/http.png differ diff --git a/tests/results/ngf-upgrade/1.3.0/https.png b/tests/results/ngf-upgrade/1.3.0/https.png new file mode 100644 index 0000000000..c738f8fd6b Binary files /dev/null and b/tests/results/ngf-upgrade/1.3.0/https.png differ diff --git a/tests/results/scale/1.3.0/1.3.0.md b/tests/results/scale/1.3.0/1.3.0.md new file mode 100755 index 0000000000..a13e18888d --- /dev/null +++ b/tests/results/scale/1.3.0/1.3.0.md @@ -0,0 +1,202 @@ +# Results + +## Test environment + +NGINX Plus: false + +GKE Cluster: + +- Node count: 12 +- k8s version: v1.28.9-gke.1000000 +- vCPUs per node: 16 +- RAM per node: 65855088Ki +- Max pods per node: 110 +- Zone: us-east1-b +- Instance Type: n2d-standard-16 + +## Summary + +- New look since we are now auto-generating these results +- Batch processing time has reduced +- Reload time is similar (slightly better in some cases) +- Metrics (CPU, memory, TTR) all seem to be better, though this could be because of the automation instead of being a manual process. + +## Test TestScale_Listeners + +### Reloads + +- Total: 126 +- Total Errors: 0 +- Average Time: 140ms +- Reload distribution: + - 500ms: 126 + - 1000ms: 126 + - 5000ms: 126 + - 10000ms: 126 + - 30000ms: 126 + - +Infms: 126 + +### Event Batch Processing + +- Total: 385 +- Average Time: 124ms +- Event Batch Processing distribution: + - 500ms: 350 + - 1000ms: 383 + - 5000ms: 385 + - 10000ms: 385 + - 30000ms: 385 + - +Infms: 385 + +### Errors + +- NGF errors: 0 +- NGF container restarts: 0 +- NGINX errors: 0 +- NGINX container restarts: 0 + +### Graphs and Logs + +See [output directory](./TestScale_Listeners) for more details. +The logs are attached only if there are errors. + +## Test TestScale_HTTPSListeners + +### Reloads + +- Total: 128 +- Total Errors: 0 +- Average Time: 165ms +- Reload distribution: + - 500ms: 128 + - 1000ms: 128 + - 5000ms: 128 + - 10000ms: 128 + - 30000ms: 128 + - +Infms: 128 + +### Event Batch Processing + +- Total: 451 +- Average Time: 115ms +- Event Batch Processing distribution: + - 500ms: 411 + - 1000ms: 450 + - 5000ms: 451 + - 10000ms: 451 + - 30000ms: 451 + - +Infms: 451 + +### Errors + +- NGF errors: 0 +- NGF container restarts: 0 +- NGINX errors: 0 +- NGINX container restarts: 0 + +### Graphs and Logs + +See [output directory](./TestScale_HTTPSListeners) for more details. +The logs are attached only if there are errors. + +## Test TestScale_HTTPRoutes + +### Reloads + +- Total: 1001 +- Total Errors: 0 +- Average Time: 363ms +- Reload distribution: + - 500ms: 756 + - 1000ms: 1001 + - 5000ms: 1001 + - 10000ms: 1001 + - 30000ms: 1001 + - +Infms: 1001 + +### Event Batch Processing + +- Total: 1008 +- Average Time: 408ms +- Event Batch Processing distribution: + - 500ms: 665 + - 1000ms: 1008 + - 5000ms: 1008 + - 10000ms: 1008 + - 30000ms: 1008 + - +Infms: 1008 + +### Errors + +- NGF errors: 0 +- NGF container restarts: 0 +- NGINX errors: 0 +- NGINX container restarts: 0 + +### Graphs and Logs + +See [output directory](./TestScale_HTTPRoutes) for more details. +The logs are attached only if there are errors. + +## Test TestScale_UpstreamServers + +### Reloads + +- Total: 117 +- Total Errors: 0 +- Average Time: 125ms +- Reload distribution: + - 500ms: 117 + - 1000ms: 117 + - 5000ms: 117 + - 10000ms: 117 + - 30000ms: 117 + - +Infms: 117 + +### Event Batch Processing + +- Total: 119 +- Average Time: 194ms +- Event Batch Processing distribution: + - 500ms: 119 + - 1000ms: 119 + - 5000ms: 119 + - 10000ms: 119 + - 30000ms: 119 + - +Infms: 119 + +### Errors + +- NGF errors: 0 +- NGF container restarts: 0 +- NGINX errors: 0 +- NGINX container restarts: 0 + +### Graphs and Logs + +See [output directory](./TestScale_UpstreamServers) for more details. +The logs are attached only if there are errors. + +## Test TestScale_HTTPMatches + +```text +Requests [total, rate, throughput] 29999, 1000.01, 999.04 +Duration [total, attack, wait] 30s, 29.999s, 933.413µs +Latencies [min, mean, 50, 90, 95, 99, max] 543.448µs, 920.709µs, 856.122µs, 988.132µs, 1.045ms, 1.255ms, 35.219ms +Bytes In [total, mean] 4799756, 160.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 99.91% +Status Codes [code:count] 200:29971 502:28 +Error Set: +502 Bad Gateway +``` +```text +Requests [total, rate, throughput] 29999, 1000.03, 1000.00 +Duration [total, attack, wait] 29.999s, 29.998s, 932.511µs +Latencies [min, mean, 50, 90, 95, 99, max] 693.08µs, 957.533µs, 941.256µs, 1.085ms, 1.133ms, 1.262ms, 3.726ms +Bytes In [total, mean] 4799840, 160.00 +Bytes Out [total, mean] 0, 0.00 +Success [ratio] 100.00% +Status Codes [code:count] 200:29999 +Error Set: +``` diff --git a/tests/results/scale/1.3.0/TestScale_HTTPRoutes/cpu.png b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/cpu.png new file mode 100644 index 0000000000..b80d220740 Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/cpu.png differ diff --git a/tests/results/scale/1.3.0/TestScale_HTTPRoutes/memory.png b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/memory.png new file mode 100644 index 0000000000..738350e52a Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/memory.png differ diff --git a/tests/results/scale/1.3.0/TestScale_HTTPRoutes/ttr.png b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/ttr.png new file mode 100644 index 0000000000..cb2a60aa6b Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPRoutes/ttr.png differ diff --git a/tests/results/scale/1.3.0/TestScale_HTTPSListeners/cpu.png b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/cpu.png new file mode 100644 index 0000000000..b9583e7282 Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/cpu.png differ diff --git a/tests/results/scale/1.3.0/TestScale_HTTPSListeners/memory.png b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/memory.png new file mode 100644 index 0000000000..d6ba0d153e Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/memory.png differ diff --git a/tests/results/scale/1.3.0/TestScale_HTTPSListeners/ttr.png b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/ttr.png new file mode 100644 index 0000000000..8ea0b8abb1 Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_HTTPSListeners/ttr.png differ diff --git a/tests/results/scale/1.3.0/TestScale_Listeners/cpu.png b/tests/results/scale/1.3.0/TestScale_Listeners/cpu.png new file mode 100644 index 0000000000..d66443fdca Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_Listeners/cpu.png differ diff --git a/tests/results/scale/1.3.0/TestScale_Listeners/memory.png b/tests/results/scale/1.3.0/TestScale_Listeners/memory.png new file mode 100644 index 0000000000..305f5a20cb Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_Listeners/memory.png differ diff --git a/tests/results/scale/1.3.0/TestScale_Listeners/ttr.png b/tests/results/scale/1.3.0/TestScale_Listeners/ttr.png new file mode 100644 index 0000000000..25fb571008 Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_Listeners/ttr.png differ diff --git a/tests/results/scale/1.3.0/TestScale_UpstreamServers/cpu.png b/tests/results/scale/1.3.0/TestScale_UpstreamServers/cpu.png new file mode 100644 index 0000000000..e27e78f31f Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_UpstreamServers/cpu.png differ diff --git a/tests/results/scale/1.3.0/TestScale_UpstreamServers/memory.png b/tests/results/scale/1.3.0/TestScale_UpstreamServers/memory.png new file mode 100644 index 0000000000..0dca37dabd Binary files /dev/null and b/tests/results/scale/1.3.0/TestScale_UpstreamServers/memory.png differ