Skip to content

Commit d46162d

Browse files
nginx-botnginx-bot
authored
NFR Test Results for NGF version 1.3.0 (#2107)
1 parent a5bfdbd commit d46162d

File tree

16 files changed

+335
-0
lines changed

16 files changed

+335
-0
lines changed

‎tests/results/dp-perf/1.3.0/1.3.0.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
GKE Cluster:
8+
9+
- Node count: 12
10+
- k8s version: v1.28.9-gke.1000000
11+
- vCPUs per node: 16
12+
- RAM per node: 65855088Ki
13+
- Max pods per node: 110
14+
- Zone: us-east1-b
15+
- Instance Type: n2d-standard-16
16+
17+
## Summary
18+
19+
- Performance seems to have improved.
20+
21+
## Test1: Running latte path based routing
22+
23+
```text
24+
Requests [total, rate, throughput] 30000, 1000.03, 1000.00
25+
Duration [total, attack, wait] 30s, 29.999s, 934.498µs
26+
Latencies [min, mean, 50, 90, 95, 99, max] 594.289µs, 940.484µs, 894.291µs, 1.076ms, 1.155ms, 1.475ms, 26.622ms
27+
Bytes In [total, mean] 4740000, 158.00
28+
Bytes Out [total, mean] 0, 0.00
29+
Success [ratio] 100.00%
30+
Status Codes [code:count] 200:30000
31+
Error Set:
32+
```
33+
34+
## Test2: Running coffee header based routing
35+
36+
```text
37+
Requests [total, rate, throughput] 30000, 1000.00, 999.97
38+
Duration [total, attack, wait] 30.001s, 30s, 885.05µs
39+
Latencies [min, mean, 50, 90, 95, 99, max] 630.729µs, 925.423µs, 909.768µs, 1.061ms, 1.117ms, 1.264ms, 10.285ms
40+
Bytes In [total, mean] 4770000, 159.00
41+
Bytes Out [total, mean] 0, 0.00
42+
Success [ratio] 100.00%
43+
Status Codes [code:count] 200:30000
44+
Error Set:
45+
```
46+
47+
## Test3: Running coffee query based routing
48+
49+
```text
50+
Requests [total, rate, throughput] 30000, 1000.04, 1000.01
51+
Duration [total, attack, wait] 30s, 29.999s, 867.406µs
52+
Latencies [min, mean, 50, 90, 95, 99, max] 649.447µs, 939.602µs, 913.399µs, 1.068ms, 1.137ms, 1.32ms, 36.366ms
53+
Bytes In [total, mean] 5010000, 167.00
54+
Bytes Out [total, mean] 0, 0.00
55+
Success [ratio] 100.00%
56+
Status Codes [code:count] 200:30000
57+
Error Set:
58+
```
59+
60+
## Test4: Running tea GET method based routing
61+
62+
```text
63+
Requests [total, rate, throughput] 30000, 1000.02, 1000.00
64+
Duration [total, attack, wait] 30s, 29.999s, 798.43µs
65+
Latencies [min, mean, 50, 90, 95, 99, max] 639.884µs, 941.213µs, 920.839µs, 1.067ms, 1.117ms, 1.266ms, 16.095ms
66+
Bytes In [total, mean] 4680000, 156.00
67+
Bytes Out [total, mean] 0, 0.00
68+
Success [ratio] 100.00%
69+
Status Codes [code:count] 200:30000
70+
Error Set:
71+
```
72+
73+
## Test5: Running tea POST method based routing
74+
75+
```text
76+
Requests [total, rate, throughput] 30000, 1000.00, 999.97
77+
Duration [total, attack, wait] 30.001s, 30s, 1.006ms
78+
Latencies [min, mean, 50, 90, 95, 99, max] 638.741µs, 951.815µs, 936.228µs, 1.085ms, 1.138ms, 1.261ms, 9.666ms
79+
Bytes In [total, mean] 4680000, 156.00
80+
Bytes Out [total, mean] 0, 0.00
81+
Success [ratio] 100.00%
82+
Status Codes [code:count] 200:30000
83+
Error Set:
84+
```

‎tests/results/ngf-upgrade/1.3.0/1.3.0.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
GKE Cluster:
8+
9+
- Node count: 12
10+
- k8s version: v1.28.9-gke.1000000
11+
- vCPUs per node: 16
12+
- RAM per node: 65855088Ki
13+
- Max pods per node: 110
14+
- Zone: us-east1-b
15+
- Instance Type: n2d-standard-16
16+
17+
## Summary
18+
19+
- Similar results as last time.
20+
21+
## Test: Send http /coffee traffic
22+
23+
```text
24+
Requests [total, rate, throughput] 6000, 100.02, 100.01
25+
Duration [total, attack, wait] 59.992s, 59.99s, 1.099ms
26+
Latencies [min, mean, 50, 90, 95, 99, max] 795.938µs, 1.153ms, 1.119ms, 1.308ms, 1.386ms, 1.658ms, 13.45ms
27+
Bytes In [total, mean] 954000, 159.00
28+
Bytes Out [total, mean] 0, 0.00
29+
Success [ratio] 100.00%
30+
Status Codes [code:count] 200:6000
31+
Error Set:
32+
```
33+
34+
![http.png](http.png)
35+
36+
## Test: Send https /tea traffic
37+
38+
```text
39+
Requests [total, rate, throughput] 6000, 100.02, 100.00
40+
Duration [total, attack, wait] 59.999s, 59.991s, 7.907ms
41+
Latencies [min, mean, 50, 90, 95, 99, max] 836.052µs, 1.21ms, 1.16ms, 1.38ms, 1.481ms, 1.842ms, 13.741ms
42+
Bytes In [total, mean] 912000, 152.00
43+
Bytes Out [total, mean] 0, 0.00
44+
Success [ratio] 100.00%
45+
Status Codes [code:count] 200:6000
46+
Error Set:
47+
```
48+
49+
![https.png](https.png)

‎tests/results/ngf-upgrade/1.3.0/http.png

4.19 KB
Loading

‎tests/results/ngf-upgrade/1.3.0/https.png

4.19 KB
Loading

‎tests/results/scale/1.3.0/1.3.0.md

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
GKE Cluster:
8+
9+
- Node count: 12
10+
- k8s version: v1.28.9-gke.1000000
11+
- vCPUs per node: 16
12+
- RAM per node: 65855088Ki
13+
- Max pods per node: 110
14+
- Zone: us-east1-b
15+
- Instance Type: n2d-standard-16
16+
17+
## Summary
18+
19+
- New look since we are now auto-generating these results
20+
- Batch processing time has reduced
21+
- Reload time is similar (slightly better in some cases)
22+
- Metrics (CPU, memory, TTR) all seem to be better, though this could be because of the automation instead of being a manual process.
23+
24+
## Test TestScale_Listeners
25+
26+
### Reloads
27+
28+
- Total: 126
29+
- Total Errors: 0
30+
- Average Time: 140ms
31+
- Reload distribution:
32+
- 500ms: 126
33+
- 1000ms: 126
34+
- 5000ms: 126
35+
- 10000ms: 126
36+
- 30000ms: 126
37+
- +Infms: 126
38+
39+
### Event Batch Processing
40+
41+
- Total: 385
42+
- Average Time: 124ms
43+
- Event Batch Processing distribution:
44+
- 500ms: 350
45+
- 1000ms: 383
46+
- 5000ms: 385
47+
- 10000ms: 385
48+
- 30000ms: 385
49+
- +Infms: 385
50+
51+
### Errors
52+
53+
- NGF errors: 0
54+
- NGF container restarts: 0
55+
- NGINX errors: 0
56+
- NGINX container restarts: 0
57+
58+
### Graphs and Logs
59+
60+
See [output directory](./TestScale_Listeners) for more details.
61+
The logs are attached only if there are errors.
62+
63+
## Test TestScale_HTTPSListeners
64+
65+
### Reloads
66+
67+
- Total: 128
68+
- Total Errors: 0
69+
- Average Time: 165ms
70+
- Reload distribution:
71+
- 500ms: 128
72+
- 1000ms: 128
73+
- 5000ms: 128
74+
- 10000ms: 128
75+
- 30000ms: 128
76+
- +Infms: 128
77+
78+
### Event Batch Processing
79+
80+
- Total: 451
81+
- Average Time: 115ms
82+
- Event Batch Processing distribution:
83+
- 500ms: 411
84+
- 1000ms: 450
85+
- 5000ms: 451
86+
- 10000ms: 451
87+
- 30000ms: 451
88+
- +Infms: 451
89+
90+
### Errors
91+
92+
- NGF errors: 0
93+
- NGF container restarts: 0
94+
- NGINX errors: 0
95+
- NGINX container restarts: 0
96+
97+
### Graphs and Logs
98+
99+
See [output directory](./TestScale_HTTPSListeners) for more details.
100+
The logs are attached only if there are errors.
101+
102+
## Test TestScale_HTTPRoutes
103+
104+
### Reloads
105+
106+
- Total: 1001
107+
- Total Errors: 0
108+
- Average Time: 363ms
109+
- Reload distribution:
110+
- 500ms: 756
111+
- 1000ms: 1001
112+
- 5000ms: 1001
113+
- 10000ms: 1001
114+
- 30000ms: 1001
115+
- +Infms: 1001
116+
117+
### Event Batch Processing
118+
119+
- Total: 1008
120+
- Average Time: 408ms
121+
- Event Batch Processing distribution:
122+
- 500ms: 665
123+
- 1000ms: 1008
124+
- 5000ms: 1008
125+
- 10000ms: 1008
126+
- 30000ms: 1008
127+
- +Infms: 1008
128+
129+
### Errors
130+
131+
- NGF errors: 0
132+
- NGF container restarts: 0
133+
- NGINX errors: 0
134+
- NGINX container restarts: 0
135+
136+
### Graphs and Logs
137+
138+
See [output directory](./TestScale_HTTPRoutes) for more details.
139+
The logs are attached only if there are errors.
140+
141+
## Test TestScale_UpstreamServers
142+
143+
### Reloads
144+
145+
- Total: 117
146+
- Total Errors: 0
147+
- Average Time: 125ms
148+
- Reload distribution:
149+
- 500ms: 117
150+
- 1000ms: 117
151+
- 5000ms: 117
152+
- 10000ms: 117
153+
- 30000ms: 117
154+
- +Infms: 117
155+
156+
### Event Batch Processing
157+
158+
- Total: 119
159+
- Average Time: 194ms
160+
- Event Batch Processing distribution:
161+
- 500ms: 119
162+
- 1000ms: 119
163+
- 5000ms: 119
164+
- 10000ms: 119
165+
- 30000ms: 119
166+
- +Infms: 119
167+
168+
### Errors
169+
170+
- NGF errors: 0
171+
- NGF container restarts: 0
172+
- NGINX errors: 0
173+
- NGINX container restarts: 0
174+
175+
### Graphs and Logs
176+
177+
See [output directory](./TestScale_UpstreamServers) for more details.
178+
The logs are attached only if there are errors.
179+
180+
## Test TestScale_HTTPMatches
181+
182+
```text
183+
Requests [total, rate, throughput] 29999, 1000.01, 999.04
184+
Duration [total, attack, wait] 30s, 29.999s, 933.413µs
185+
Latencies [min, mean, 50, 90, 95, 99, max] 543.448µs, 920.709µs, 856.122µs, 988.132µs, 1.045ms, 1.255ms, 35.219ms
186+
Bytes In [total, mean] 4799756, 160.00
187+
Bytes Out [total, mean] 0, 0.00
188+
Success [ratio] 99.91%
189+
Status Codes [code:count] 200:29971 502:28
190+
Error Set:
191+
502 Bad Gateway
192+
```
193+
```text
194+
Requests [total, rate, throughput] 29999, 1000.03, 1000.00
195+
Duration [total, attack, wait] 29.999s, 29.998s, 932.511µs
196+
Latencies [min, mean, 50, 90, 95, 99, max] 693.08µs, 957.533µs, 941.256µs, 1.085ms, 1.133ms, 1.262ms, 3.726ms
197+
Bytes In [total, mean] 4799840, 160.00
198+
Bytes Out [total, mean] 0, 0.00
199+
Success [ratio] 100.00%
200+
Status Codes [code:count] 200:29999
201+
Error Set:
202+
```

‎tests/results/scale/1.3.0/TestScale_HTTPRoutes/cpu.png

7.92 KB
Loading

‎tests/results/scale/1.3.0/TestScale_HTTPRoutes/memory.png

6.71 KB
Loading

‎tests/results/scale/1.3.0/TestScale_HTTPRoutes/ttr.png

6.42 KB
Loading

‎tests/results/scale/1.3.0/TestScale_HTTPSListeners/cpu.png

6.76 KB
Loading

‎tests/results/scale/1.3.0/TestScale_HTTPSListeners/memory.png

5.85 KB
Loading

‎tests/results/scale/1.3.0/TestScale_HTTPSListeners/ttr.png

8.78 KB
Loading

‎tests/results/scale/1.3.0/TestScale_Listeners/cpu.png

6.55 KB
Loading

‎tests/results/scale/1.3.0/TestScale_Listeners/memory.png

6.06 KB
Loading

‎tests/results/scale/1.3.0/TestScale_Listeners/ttr.png

9.33 KB
Loading

‎tests/results/scale/1.3.0/TestScale_UpstreamServers/cpu.png

6.3 KB
Loading

‎tests/results/scale/1.3.0/TestScale_UpstreamServers/memory.png

5.18 KB
Loading

0 commit comments

Comments
 (0)