Skip to content

Commit e544179

Browse files
sjbermanamimimor
sjberman
authored and
amimimor
committed
Reconfig test results for 1.2.0 (nginx#1718)
Add reconfig test results for the 1.2.0 release
1 parent c953c5d commit e544179

File tree

2 files changed

+107
-1
lines changed

2 files changed

+107
-1
lines changed

‎tests/reconfig/results/1.2.0/1.2.0.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Reconfiguration testing Results
2+
3+
<!-- TOC -->
4+
- [Reconfiguration testing Results](#reconfiguration-testing-results)
5+
- [Summary](#summary)
6+
- [Test environment](#test-environment)
7+
- [Results Tables](#results-tables)
8+
- [NGINX Reloads and Time to Ready](#nginx-reloads-and-time-to-ready)
9+
- [Event Batch Processing](#event-batch-processing)
10+
- [NumResources to Total Resources](#numresources-to-total-resources)
11+
- [Observations](#observations)
12+
- [Future Improvements](#future-improvements)
13+
<!-- TOC -->
14+
15+
## Summary
16+
17+
- Time to ready stayed consistent, if not slightly faster.
18+
- Reload time has slightly increased in some instances.
19+
- Number of batch events has reduced, subsequently increasing the average time of each batch.
20+
21+
## Test environment
22+
23+
GKE cluster:
24+
25+
- Node count: 3
26+
- Instance Type: e2-medium
27+
- k8s version: 1.27.8-gke.1067004
28+
- Zone: us-west2-a
29+
- Total vCPUs: 6
30+
- Total RAM: 12GB
31+
- Max pods per node: 110
32+
33+
NGF deployment:
34+
35+
- NGF version: edge - git commit 96a44240d317875406a8aef8fd1e424f2fb906eb
36+
- NGINX OSS Version: 1.25.4
37+
- NGINX Plus Version: R31
38+
39+
## Results Tables
40+
41+
> Note: After fixing the `cafe-tls-redirect` to point to the proper Gateway, tests that created 450 HTTPRoutes failed due to https://github.com/nginxinc/nginx-gateway-fabric/issues/1107. Therefore, those tests were re-run after reverting the `cafe-tls-redirect` issue to maintain consistency with the previous release tests. Going forward, results should look different once the above bug is fixed. Added N+ tests, but without testing 150 since it has the bug mentioned above.
42+
43+
### NGINX Reloads and Time to Ready
44+
45+
#### OSS
46+
47+
| Test number | NumResources | TimeToReadyTotal (s) | TimeToReadyAvgSingle (s) | NGINX reloads | NGINX reload avg time (ms) | <= 500ms | <= 1000ms |
48+
|-------------|--------------|----------------------|--------------------------|---------------|----------------------------|----------|-----------|
49+
| 1 | 30 | 2 | <1 | 2 | 189.5 | 100% | 100% |
50+
| 1 | 150 | 2 | <1 | 2 | 389 | 100% | 100% |
51+
| 2 | 30 | 30 | <1 | 94 | 161 | 100% | 100% |
52+
| 2 | 150 | 154 | <1 | 387 | 267.48 | 100% | 100% |
53+
| 3 | 30 | <1 | <1 | 94 | 127.91 | 100% | 100% |
54+
| 3 | 150 | <1 | <1 | 454 | 128 | 100% | 100% |
55+
56+
#### Plus
57+
58+
| Test number | NumResources | TimeToReadyTotal (s) | TimeToReadyAvgSingle (s) | NGINX reloads | NGINX reload avg time (ms) | <= 500ms | <= 1000ms |
59+
|-------------|--------------|----------------------|--------------------------|---------------|----------------------------|----------|-----------|
60+
| 1 | 30 | 1 | <1 | 2 | 151.5 | 100% | 100% |
61+
| 2 | 30 | 30 | <1 | 94 | 157 | 100% | 100% |
62+
| 3 | 30 | <1 | <1 | 94 | 128 | 100% | 100% |
63+
64+
### Event Batch Processing
65+
66+
#### OSS
67+
68+
| Test number | NumResources | Event Batch Total | Event Batch Processing avg time (ms) | <= 500ms | <= 1000ms | <= 5000ms | <= 10000ms | <= 30000ms |
69+
|-------------|--------------|-------------------|--------------------------------------|----------|-----------|-----------|------------|------------|
70+
| 1 | 30 | 5 | 733.6 | 80% | 80% | 100% | 100% | 100% |
71+
| 1 | 150 | 5 | 2967 | 40% | 40% | 40% | 40% | 40% |
72+
| 2 | 30 | 371 | 57.32 | 100% | 100% | 100% | 100% | 100% |
73+
| 2 | 150 | 1743 | 75.87 | 98.45% | 100% | 100% | 100% | 100% |
74+
| 3 | 30 | 370 | 37.48 | 99.73% | 99.73% | 100% | 100% | 100% |
75+
| 3 | 150 | 1808 | 40.18 | 99.94% | 99.94% | 99.94% | 99.94% | 100% |
76+
77+
#### Plus
78+
79+
| Test number | NumResources | Event Batch Total | Event Batch Processing avg time (ms) | <= 500ms | <= 1000ms | <= 5000ms | <= 10000ms | <= 30000ms |
80+
|-------------|--------------|-------------------|--------------------------------------|----------|-----------|-----------|------------|------------|
81+
| 1 | 30 | 3 | 1170 | 66% | 66% | 100% | 100% | 100% |
82+
| 2 | 30 | 370 | 58.79 | 100% | 100% | 100% | 100% | 100% |
83+
| 3 | 30 | 370 | 41.32 | 99.73% | 99.73% | 100% | 100% | 100% |
84+
85+
## NumResources to Total Resources
86+
87+
| NumResources | Gateways | Secrets | ReferenceGrants | Namespaces | application Pods | application Services | HTTPRoutes | Attached HTTPRoutes | Total Resources |
88+
|--------------|----------|---------|-----------------|------------|------------------|----------------------|------------|---------------------|-----------------|
89+
| x | 1 | 1 | 1 | x+1 | 2x | 2x | 3x | 2x | <total> |
90+
| 30 | 1 | 1 | 1 | 31 | 60 | 60 | 90 | 60 | 244 |
91+
| 150 | 1 | 1 | 1 | 151 | 300 | 300 | 450 | 300 | 1204 |
92+
93+
> Note: Only 2x HTTPRoutes attach to the Gateway because the parentRef name in the `cafe-tls-redirect` HTTPRoute is incorrect. This has been fixed, but until https://github.com/nginxinc/nginx-gateway-fabric/issues/1107 is fixed we can't actually run the test successfully.
94+
95+
## Observations
96+
97+
1. Reload time seems to have a increased slightly in a few instances, though time to ready is consistent if not faster.
98+
99+
2. Processing fewer batches overall due to improvements in resource event tracking. Overall processing time didn't change much, so the average increased due to fewer batches.
100+
101+
3. No errors in the logs.
102+
103+
104+
## Future Improvements
105+
106+
Fix https://github.com/nginxinc/nginx-gateway-fabric/issues/1107 to allow for 150 resource tests to properly run.

‎tests/reconfig/scripts/cafe-routes.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ metadata:
44
name: cafe-tls-redirect
55
spec:
66
parentRefs:
7-
- name: gateway.networking.k8s.io/v1
7+
- name: gateway
88
namespace: default
99
sectionName: http
1010
hostnames:

0 commit comments

Comments
 (0)