From 8685039523df2268c8885019a21fc9c0a362156c Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Tue, 3 Jun 2025 14:45:02 -0700 Subject: [PATCH 1/7] add connection pools page and example --- .../connection-options/connection-pools.txt | 157 +++++++++++++++++- .../includes/connect/connection-pools-uri.go | 28 ++++ 2 files changed, 184 insertions(+), 1 deletion(-) create mode 100644 source/includes/connect/connection-pools-uri.go diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index c4618800..e636c118 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -4,4 +4,159 @@ Connection Pools ================ -.. TODO \ No newline at end of file +.. contents:: On this page + :local: + :backlinks: none + :depth: 2 + :class: singlecol + +.. facet:: + :name: genre + :values: reference + +Overview +-------- + +In this guide, you can learn about how the {+driver-short+} uses connection +pools to manage connections to a MongoDB deployment and how you can configure +connection pool settings in your application. + +A connection pool is a cache of open database connections maintained by the +{+driver-short+}. When your application requests a connection to MongoDB, the +{+driver-short+} seamlessly gets a connection from the pool, performs +operations, and returns the connection to the pool for reuse. + +Connection pools help reduce application latency and the number of times new +connections are created by {+driver-short+}. + +.. _golang-faq-connection-pool: + +Create a Connection Pool +------------------------ + +Every ``Client`` instance has a built-in connection pool for each server in your +MongoDB topology. If you do not cofigure the ``minPoolSize`` option, connection +pools open sockets on demand. These sockets support concurrent MongoDB +operations, or `goroutines `__, in +your application. + +Each ``Client`` instance opens two more sockets per server in your MongoDB +topology for monitoring the server's state. + +For example, a client connected to a three-node replica set opens six monitoring +sockets. If the application uses the default setting for ``maxPoolSize`` and +only queries the primary (default) node, then there can be at most ``106`` open +sockets and ``100`` connections in the connection pool. If the application uses +a :ref:`read preference ` to query the secondary nodes, their +pools also grow and there can be ``306`` total connections. + +Create a client once for each process, and reuse it for all operations. Avoid +creating a new client for each request, as this is inefficient. + +Configure a Connection Pool +--------------------------- + +You can specify settings for your connection pool either by using a connection +string with the ``options.Client().ApplyURI()`` method. + +The following are connection string settings you can use to configure your +connection pool: + +.. list-table:: + :widths: 25,75 + :header-rows: 1 + + * - Setting + - Description + + * - ``maxPoolSize`` + + - Maximum number of connections opened in the pool. If an operation needs a + new connection while the connection pool has ``maxPoolSize`` connections + open, the new operation waits for a new connection to open. To limit this + waiting time, use the single timeout setting. + + *Default:* ``100`` + + * - ``minPoolSize`` + + - Minimum number of connections opened in the pool. The value of + ``minPoolSize`` must be less than the value of ``maxPoolSize``. + + *Default*: ``0`` + + * - ``maxConnecting`` + + - Maximum number of connections a pool may establish concurrently. + + *Default:* ``2`` + + * - ``maxIdleTimeMS`` + + - The maximum number of milliseconds that a connection can remain idle in + the pool before being removed and closed. + + *Default:* ``None`` (no limit) + + * - ``waitQueueTimeoutMS``` + + - Specifies the maximum wait time in milliseconds that a thread can wait + for a connection to become available. + + *Default*: ``0`` (no limit) + +The following code creates a client with a maximum connection pool size of +``50``, a minimum pool size of ``10``, and a maximum idle time of +``30000`` milliseconds (30 seconds): + +.. literalinclude:: /includes/connect/connection-pools-uri.go + :language: go + :start-after: start-connection-pool-uri + :end-before: end-connection-pool-uri + :dedent: + +maxPoolSize +~~~~~~~~~~~ + +To suppor high numbers of concurrent operations, you can increase +``maxPoolSize``. Once the pool reaches its maximum size, additional operations +wait for sockets to become availble. + +minPoolSize +~~~~~~~~~~~ + +The connection pool initializes with the number of sockets specified by +``minPoolSize``. If the total number of sockets drops below this minimum, more +sockets open until the minimum is reached. + +maxConnecting +~~~~~~~~~~~~~ + +This option limits the number of connections that a deployment can create +simultaneously. This helps manage resource usage and improve the driver's +ability to reuse existing connections. + +maxIdleTimeMS +~~~~~~~~~~~~~ + +Use this option to specify how long a connection can remain idle before being +removed. This helps manage resource usage by closing unused connections. + +waitQueueTimeoutMS +~~~~~~~~~~~~~~~~~~ + +This option allows you to specify how long an operation can wait for a +connection. This is useful for managing operation duration during load spikes. + +Disconnecting +------------- + +When you call ``Client.Disconnect()`` from any goroutine, the driver closes all +idle sockets and closes all sockets in use as they are returned to the pool. + +Additional Information +---------------------- + +For more information on using a connection pool, see the :manual:`Connection +Pool ` documentation in the Server +manual. \ No newline at end of file diff --git a/source/includes/connect/connection-pools-uri.go b/source/includes/connect/connection-pools-uri.go new file mode 100644 index 00000000..70d9042b --- /dev/null +++ b/source/includes/connect/connection-pools-uri.go @@ -0,0 +1,28 @@ +package main + +import ( + "context" + "log" + + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +func main() { + //start-connection-pool-uri + // Connection string with connection pool options + uri := "mongodb://localhost:27017/?maxPoolSize=50&minPoolSize=10&maxIdleTimeMS=30000" + + // Creates a new client and connect to the server + client, err := mongo.Connect(options.Client().ApplyURI(uri)) + if err != nil { + log.Fatal(err) + } + //end-connection-pool-uri + + defer func() { + if err = client.Disconnect(context.TODO()); err != nil { + log.Fatal(err) + } + }() +} From 6bd4655ec1420691ab0037e2485289be01e05a69 Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Tue, 3 Jun 2025 15:03:09 -0700 Subject: [PATCH 2/7] edits --- .../connection-options/connection-pools.txt | 53 ++++++++----------- 1 file changed, 21 insertions(+), 32 deletions(-) diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index e636c118..d5b5bf8e 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -115,38 +115,27 @@ The following code creates a client with a maximum connection pool size of :end-before: end-connection-pool-uri :dedent: -maxPoolSize -~~~~~~~~~~~ - -To suppor high numbers of concurrent operations, you can increase -``maxPoolSize``. Once the pool reaches its maximum size, additional operations -wait for sockets to become availble. - -minPoolSize -~~~~~~~~~~~ - -The connection pool initializes with the number of sockets specified by -``minPoolSize``. If the total number of sockets drops below this minimum, more -sockets open until the minimum is reached. - -maxConnecting -~~~~~~~~~~~~~ - -This option limits the number of connections that a deployment can create -simultaneously. This helps manage resource usage and improve the driver's -ability to reuse existing connections. - -maxIdleTimeMS -~~~~~~~~~~~~~ - -Use this option to specify how long a connection can remain idle before being -removed. This helps manage resource usage by closing unused connections. - -waitQueueTimeoutMS -~~~~~~~~~~~~~~~~~~ - -This option allows you to specify how long an operation can wait for a -connection. This is useful for managing operation duration during load spikes. +Optimize Connection Pools +------------------------- + +Connection pools are rate-limited such that each connection pool can only create, +at maximum, the value of ``maxConnecting`` connections in parallel at any time. +Any new goroutine stops waiting in the following cases: + +- One of the existing goroutines finishes creating a connection, or an existing + connection is checked back into the pool. +- The driver's ability to reuse existing connections improves due to rate-limits + on connection creation. + +The driver does not limit the number of operations that can wait for sockets to +become available and it is the application's responsibility to limit the size of +its pool to bound queuing during a load spike. Operations can wait for any length of time +unless you define the ``waitQueueTimeoutMS`` option. + +An operation that waits more than the length of time defined by +``waitQueueTimeoutMS`` for a socket raises a connection error. Use this option +if it is more important to bound the duration of operations during a load spike +than it is to complete every operation. Disconnecting ------------- From 471a9e57ce1e5826e7089703211f5f4899cb4efe Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Tue, 3 Jun 2025 15:40:04 -0700 Subject: [PATCH 3/7] fix and add links --- .../connection-options/connection-pools.txt | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index d5b5bf8e..086538fe 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -35,9 +35,9 @@ Create a Connection Pool ------------------------ Every ``Client`` instance has a built-in connection pool for each server in your -MongoDB topology. If you do not cofigure the ``minPoolSize`` option, connection +MongoDB topology. If you do not configure the ``minPoolSize`` option, connection pools open sockets on demand. These sockets support concurrent MongoDB -operations, or `goroutines `__, in +operations, or `goroutines `__, in your application. Each ``Client`` instance opens two more sockets per server in your MongoDB @@ -148,4 +148,13 @@ Additional Information For more information on using a connection pool, see the :manual:`Connection Pool ` documentation in the Server -manual. \ No newline at end of file +manual. + +API Documentation +~~~~~~~~~~~~~~~~~ + +To learn more about any of the methods or types discussed in this +guide, see the following API documentation: + +- `Client <{+api+}/mongo#Client>`__ +- `Client.Diconnect() <{+api+}/mongo#Client.Disconnect>`__ \ No newline at end of file From 962a81386490c8046866611ff742890c2d259dcd Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Tue, 3 Jun 2025 16:22:35 -0700 Subject: [PATCH 4/7] add api doc link --- source/connect/connection-options/connection-pools.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index 086538fe..0a985435 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -157,4 +157,5 @@ To learn more about any of the methods or types discussed in this guide, see the following API documentation: - `Client <{+api+}/mongo#Client>`__ +- `ClientOptions <{+api+}/mongo/options#ClientOptions>`__ - `Client.Diconnect() <{+api+}/mongo#Client.Disconnect>`__ \ No newline at end of file From 666aaa1a253ec848c7e55d655e65eca2dd2b44af Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Wed, 4 Jun 2025 11:19:17 -0700 Subject: [PATCH 5/7] rm feedback --- .../connection-options/connection-pools.txt | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index 0a985435..23b907c5 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -40,8 +40,8 @@ pools open sockets on demand. These sockets support concurrent MongoDB operations, or `goroutines `__, in your application. -Each ``Client`` instance opens two more sockets per server in your MongoDB -topology for monitoring the server's state. +When a new ``Client`` instance is instatiated, it opens two sockets per server +in your MongoDB topology for monitoring the server's state. For example, a client connected to a three-node replica set opens six monitoring sockets. If the application uses the default setting for ``maxPoolSize`` and @@ -50,8 +50,9 @@ sockets and ``100`` connections in the connection pool. If the application uses a :ref:`read preference ` to query the secondary nodes, their pools also grow and there can be ``306`` total connections. -Create a client once for each process, and reuse it for all operations. Avoid -creating a new client for each request, as this is inefficient. +For efficiency, create a client once for each process, and reuse it for all +operations. Avoid creating a new client for each request because this will +increase latency. Configure a Connection Pool --------------------------- @@ -105,6 +106,9 @@ connection pool: *Default*: ``0`` (no limit) +Example +~~~~~~~ + The following code creates a client with a maximum connection pool size of ``50``, a minimum pool size of ``10``, and a maximum idle time of ``30000`` milliseconds (30 seconds): @@ -128,9 +132,9 @@ Any new goroutine stops waiting in the following cases: on connection creation. The driver does not limit the number of operations that can wait for sockets to -become available and it is the application's responsibility to limit the size of -its pool to bound queuing during a load spike. Operations can wait for any length of time -unless you define the ``waitQueueTimeoutMS`` option. +become available, so it is the application's responsibility to manage its +operation queue. Operations can wait for any length of time unless you define +the ``waitQueueTimeoutMS`` option. An operation that waits more than the length of time defined by ``waitQueueTimeoutMS`` for a socket raises a connection error. Use this option @@ -141,7 +145,7 @@ Disconnecting ------------- When you call ``Client.Disconnect()`` from any goroutine, the driver closes all -idle sockets and closes all sockets in use as they are returned to the pool. +idle sockets, and then closes all sockets as they are returned to the pool. Additional Information ---------------------- From 39c32459086d64ac9b37aef0c12fe57e7ac2e602 Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Thu, 5 Jun 2025 16:27:12 -0700 Subject: [PATCH 6/7] pv feedback --- .../connection-options/connection-pools.txt | 166 +++++++++++++----- .../connection-pools-client-options.go | 39 ++++ .../includes/connect/connection-pools-uri.go | 15 +- 3 files changed, 173 insertions(+), 47 deletions(-) create mode 100644 source/includes/connect/connection-pools-client-options.go diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index 23b907c5..6a243e6c 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -60,64 +60,146 @@ Configure a Connection Pool You can specify settings for your connection pool either by using a connection string with the ``options.Client().ApplyURI()`` method. -The following are connection string settings you can use to configure your -connection pool: +Select the :guilabel:`Connection String` or :guilabel:`MongoClientSettings` tab to +see the corresponding syntax: -.. list-table:: - :widths: 25,75 - :header-rows: 1 +.. tabs:: - * - Setting - - Description - - * - ``maxPoolSize`` + .. tab:: Connection String + :tabid: uri + + The following are connection string settings you can use to configure your + connection pool: - - Maximum number of connections opened in the pool. If an operation needs a - new connection while the connection pool has ``maxPoolSize`` connections - open, the new operation waits for a new connection to open. To limit this - waiting time, use the single timeout setting. + .. list-table:: + :widths: 25,75 + :header-rows: 1 - *Default:* ``100`` + * - Setting + - Description + + * - ``maxPoolSize`` - * - ``minPoolSize`` + - Maximum number of connections opened in the pool. If an operation needs a + new connection while the connection pool has ``maxPoolSize`` connections + open, the new operation waits for a new connection to open. To limit this + waiting time, use the single timeout setting. - - Minimum number of connections opened in the pool. The value of - ``minPoolSize`` must be less than the value of ``maxPoolSize``. + *Default:* ``100`` - *Default*: ``0`` + * - ``minPoolSize`` - * - ``maxConnecting`` - - - Maximum number of connections a pool may establish concurrently. - - *Default:* ``2`` - - * - ``maxIdleTimeMS`` - - - The maximum number of milliseconds that a connection can remain idle in - the pool before being removed and closed. + - Minimum number of connections opened in the pool. The value of + ``minPoolSize`` must be less than the value of ``maxPoolSize``. - *Default:* ``None`` (no limit) + *Default*: ``0`` - * - ``waitQueueTimeoutMS``` + * - ``maxConnecting`` + + - Maximum number of connections a pool may establish concurrently. + + *Default:* ``2`` + + * - ``maxIdleTimeMS`` + + - The maximum number of milliseconds that a connection can remain idle in + the pool before being removed and closed. - - Specifies the maximum wait time in milliseconds that a thread can wait - for a connection to become available. + *Default:* ``None`` (no limit) - *Default*: ``0`` (no limit) + * - ``waitQueueTimeoutMS``` + + - Specifies the maximum wait time in milliseconds that a thread can wait + for a connection to become available. + + *Default*: ``0`` (no limit) + + .. tab:: ClientOptions + :tabid: ClientOptions + + You can set the connection pool options directly using the ``options.Client`` methods. + + The following table describes the methods you can chain to your settings + to modify the driver's behavior: + + .. list-table:: + :widths: 25,75 + :header-rows: 1 + + * - Setting + - Description + + * - ``SetMaxPoolSize()`` + + - Maximum number of connections opened in the pool. If an operation needs a + new connection while the connection pool has the configured maximum connections + open, the new operation waits for a new connection to open. To limit this + waiting time, use the single timeout setting. + + *Default:* ``100`` + + * - ``SetMinPoolSize()`` + + - Minimum number of connections opened in the pool. The value of + ``MinPoolSize`` must be less than the value of ``MaxPoolSize``. + + *Default*: ``0`` + + * - ``SetMaxConnecting()`` + + - Maximum number of connections a pool may establish concurrently. + + *Default:* ``2`` + + * - ``SetMAxConnIdleTime()`` + + - The maximum number of milliseconds that a connection can remain idle in + the pool before being removed and closed. + + *Default:* ``0`` (no limit) Example ~~~~~~~ -The following code creates a client with a maximum connection pool size of -``50``, a minimum pool size of ``10``, and a maximum idle time of -``30000`` milliseconds (30 seconds): - -.. literalinclude:: /includes/connect/connection-pools-uri.go - :language: go - :start-after: start-connection-pool-uri - :end-before: end-connection-pool-uri - :dedent: +Select the :guilabel:`Connection String` or :guilabel:`MongoClientSettings` tab to +see the corresponding example: + +.. tabs:: + + .. tab:: Connection String + :tabid: uriExample + + The following code uses the connection string to configure the maximum + connection pool size of ``50``, a minimum pool size of ``10``, and a + maximum idle time of ``30000`` milliseconds (30 seconds): + + .. literalinclude:: /includes/connect/connection-pools-uri.go + :language: go + :start-after: start-uri-variable + :end-before: end-uri-variable + :dedent: + + The following code creates a client and passes the connection string to the + ``ApplyURI()`` method: + + .. literalinclude:: /includes/connect/connection-pools-uri.go + :language: go + :start-after: start-apply-uri + :end-before: end-apply-uri + :dedent: + + .. tab:: ClientOptions + :tabid: optionsExample + + The following code creates a client and sets the connection pool options + with a maximum connection pool size of ``50``, a minimum pool size of + ``10``, and a maximum idle time of ``30000`` milliseconds (30 seconds): + + .. literalinclude:: /includes/connect/connection-pools-client-options.go + :language: go + :start-after: start-client-options + :end-before: end-client-options + :dedent: Optimize Connection Pools ------------------------- diff --git a/source/includes/connect/connection-pools-client-options.go b/source/includes/connect/connection-pools-client-options.go new file mode 100644 index 00000000..98a4fc48 --- /dev/null +++ b/source/includes/connect/connection-pools-client-options.go @@ -0,0 +1,39 @@ +package main + +import ( + "context" + "log" + "os" + "time" + + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +func main() { + uri := os.Getenv("MONGODB_URI") + if uri == "" { + log.Fatal("Set your 'MONGODB_URI' environment variable.") + } + // start-client-options + // Sets client options with connection pool settings + clientOptions := options.Client(). + ApplyURI(uri). + SetMaxPoolSize(50). + SetMinPoolSize(10). + SetMaxConnIdleTime(30 * time.Second) + + // Creates a new client and connects to the server + client, err := mongo.Connect(clientOptions) + if err != nil { + log.Fatal(err) + } + // end-client-options + + // Ensures the client is disconnected when the function exits + defer func() { + if err = client.Disconnect(context.TODO()); err != nil { + log.Fatal(err) + } + }() +} diff --git a/source/includes/connect/connection-pools-uri.go b/source/includes/connect/connection-pools-uri.go index 70d9042b..7ae0da32 100644 --- a/source/includes/connect/connection-pools-uri.go +++ b/source/includes/connect/connection-pools-uri.go @@ -8,18 +8,23 @@ import ( "go.mongodb.org/mongo-driver/v2/mongo/options" ) -func main() { - //start-connection-pool-uri - // Connection string with connection pool options - uri := "mongodb://localhost:27017/?maxPoolSize=50&minPoolSize=10&maxIdleTimeMS=30000" +// start-uri-variable +// Connection string with connection pool options +const ( + uri = "mongodb://localhost:27017/?maxPoolSize=50&minPoolSize=10&maxIdleTimeMS=30000" +) +// end-uri-variable +func main() { + // start-apply-uri // Creates a new client and connect to the server client, err := mongo.Connect(options.Client().ApplyURI(uri)) if err != nil { log.Fatal(err) } - //end-connection-pool-uri + // end-apply-uri + fmt.log("Connected to MongoDB with connection pool options") defer func() { if err = client.Disconnect(context.TODO()); err != nil { log.Fatal(err) From 24581a3310b1a8a9490235d3cbab2d8204ceabe9 Mon Sep 17 00:00:00 2001 From: Stephanie Aurelio Date: Thu, 5 Jun 2025 16:43:15 -0700 Subject: [PATCH 7/7] edits --- source/connect/connection-options/connection-pools.txt | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/source/connect/connection-options/connection-pools.txt b/source/connect/connection-options/connection-pools.txt index 6a243e6c..1da14558 100644 --- a/source/connect/connection-options/connection-pools.txt +++ b/source/connect/connection-options/connection-pools.txt @@ -58,7 +58,7 @@ Configure a Connection Pool --------------------------- You can specify settings for your connection pool either by using a connection -string with the ``options.Client().ApplyURI()`` method. +string or by using the ``options.Client`` methods. Select the :guilabel:`Connection String` or :guilabel:`MongoClientSettings` tab to see the corresponding syntax: @@ -117,8 +117,6 @@ see the corresponding syntax: .. tab:: ClientOptions :tabid: ClientOptions - You can set the connection pool options directly using the ``options.Client`` methods. - The following table describes the methods you can chain to your settings to modify the driver's behavior: @@ -151,7 +149,7 @@ see the corresponding syntax: *Default:* ``2`` - * - ``SetMAxConnIdleTime()`` + * - ``SetMaxConnIdleTime()`` - The maximum number of milliseconds that a connection can remain idle in the pool before being removed and closed.