Static values for worker_connections lead to error

[f0rkz]

Greetings and salutations!

I wanted to report an issue we are seeing while load testing through the gateway fabric. We were receiving errors regarding not enough worker_connections.

2025/05/10 04:01:21 [warn] 1180#1180: 1024 worker_connections are not enough, reusing connections

I noticed this value is set as a static value of 1024 in the following file: https://github.com/nginx/nginx-gateway-fabric/blob/main/internal/mode/static/nginx/conf/nginx.conf#L9

Is 1024 set as a limit due to nginx-gateway-fabric's architecture?

Is this warning safe to ignore? It seems to indicate it is reusing connections in its pool.

[sjberman]

Hey @f0rkz, 1024 is not a limit for us, just a common value that we are using. It's probably something that could be made configurable.

I'm curious if you're seeing any traffic dropped or being affected? If not, then it may be okay since it's just a warning, though I can't speak for the behavior if the load increases and the issue also worsens.

Also, have you increased the number of replicas you are running? That may help distribute connections to more nginx instances and avoid this.

Another potential workaround is to edit the base nginx.conf and build a custom container image to use with that conf.

There is also a blog that describes how you can mount a custom base nginx config into the Deployment, but we are in the process of changing our architecture to split the controller and nginx into separate pods for our upcoming 2.0 release, so this blog approach may not continue to work at that point.

[f0rkz]

Thanks for the reply @sjberman. We eventually did scale out to to accommodate the larger load tests going on. That seemed to help the situation for sure but we went from a few nginx-gateway pods to ~200 replicas to accommodate the worker_connections overflow. I think there is some opportunity to make that parameter configurable as a tunable parameter for sure to ensure the replicas are smaller and doing more work instead of the massive fleet doing very little work :).

We started going down the route of editing that base nginx.conf file like you suggested but we were hesitant due to potential changes that could be pushed in the future.

Would love to keep in touch. This gateway fabric looks very promising!

[sjberman]

Thanks for the feedback. I'll create an issue for making the field configurable, and we'll figure out its priority in our roadmap.

Sidenote, keep an eye out for our 2.0 release within the next month or so. With the changing architecture, scaling nginx will be a slightly different process than it is today. We'll have documentation on how to do that.

[sjberman]

#3371

[f0rkz]

Thanks for taking this under consideration!