Skip to content

DOCSP-50497 Node.js Aggregation Standardization #1162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
286 changes: 70 additions & 216 deletions source/aggregation.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,245 +14,99 @@ Aggregation

.. contents:: On this page
:local:

:backlinks: none
:depth: 2
:class: singlecol

.. toctree::

Pipeline Stages </aggregation/pipeline-stages>
Filtered Subset </aggregation/filtered-subset/>
Group & Total </aggregation/group-total/>
Unpack Arrays & Group </aggregation/unpack-arrays/>
One-to-One Join </aggregation/one-to-one-join/>
Multi-Field Join </aggregation/multi-field-join/>

.. _nodejs-aggregation-overview:

Overview
--------

In this guide, you can learn how to use **aggregation operations** in
the MongoDB Node.js driver.
In this guide, you can learn how to use the MongoDB Node.js Driver to perform
**aggregation operations**.

Aggregation operations process data in your MongoDB collections and return
computed results. The MongoDB Aggregation framework is modeled on the
concept of data processing pipelines. Documents enter a pipeline comprised of one or
more stages, and this pipeline transforms the documents into an aggregated result.

Aggregation operations are expressions you can use to produce reduced
and summarized results in MongoDB. MongoDB's aggregation framework
allows you to create a pipeline that consists of one or more stages,
each of which performs a specific operation on your data.
To learn more about the aggregation stages supported by the Node.js Driver, see :ref:`Aggregation Stages <node-aggregation-pipeline-stages>`.

Analogy
~~~~~~~

You can think of the aggregation pipeline as similar to an automobile factory.
Automobile manufacturing requires the use of assembly stations organized
into assembly lines. Each station has specialized tools, such as
drills and welders. The factory transforms and
assembles the initial parts and materials into finished products.

The **aggregation pipeline** is the assembly line, **aggregation
stages** are the assembly stations, and **expression operators** are the
specialized tools.

Comparing Aggregation and Query Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using query operations, such as the ``find()`` method, you can perform the following actions:

- Select *which documents* to return
- Select *which fields* to return
- Sort the results

Using aggregation operations, you can perform the following actions:

- Perform all query operations
- Rename fields
- Calculate fields
- Summarize data
- Group values

Aggregation operations have some :manual:`limitations </core/aggregation-pipeline-limits/>`:

- Returned documents must not violate the :manual:`BSON-document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
The aggregation pipeline is similar to an automobile factory assembly line. An
assembly lines has stations with specialized tools that are used to perform
specific tasks. For example, when building a car, the assembly line begins with
a frame. As the car frame moves though the assembly line, each station adds a
new part. The factory transforms and assembles the initial parts, resulting in
finished cars.

The *aggregation pipeline* is the assembly line, the *aggregation stages* are
the assembly stations, and the *expression operators* are the specialized tools.

Compare Aggregation and Find Operations
---------------------------------------

The following table lists the different tasks you can perform with find
operations compared to what you can achieve with aggregation
operations. The aggregation framework provides expanded functionality
that allows you to transform and manipulate your data.

.. list-table::
:header-rows: 1
:widths: 50 50

* - Find Operations
- Aggregation Operations

* - | Select *certain* documents to return
| Select *which* fields to return
| Sort the results
| Limit the results
| Count the results
- | Select *certain* documents to return
| Select *which* fields to return
| Sort the results
| Limit the results
| Count the results
| Group the results
| Rename fields
| Compute new fields
| Summarize data
| Connect and merge data sets

Server Limitations
------------------

Consider the following :manual:`limitations </core/aggregation-pipeline-limits/>` when performing aggregation operations:

- Returned documents must not violate the :manual:`BSON document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
of 16 megabytes.

- Pipeline stages have a memory limit of 100 megabytes by default. You can exceed this
limit by setting the ``allowDiskUse`` property of ``AggregateOptions`` to ``true``. See
the `AggregateOptions API documentation <{+api+}/interfaces/AggregateOptions.html>`__
for more details.

.. important:: $graphLookup exception

The :manual:`$graphLookup
</reference/operator/aggregation/graphLookup/>` stage has a strict
memory limit of 100 megabytes and will ignore ``allowDiskUse``.

References
~~~~~~~~~~

To view a full list of expression operators, see :manual:`Aggregation
Operators </reference/operator/aggregation/>` in the Server manual.

To learn about assembling an aggregation pipeline and view examples, see
:manual:`Aggregation Pipeline </core/aggregation-pipeline/>` in the
Server manual.

To learn more about creating pipeline stages, see :manual:`Aggregation
Stages </reference/operator/aggregation-pipeline/>` in the Server manual.

Runnable Examples
-----------------

The example uses sample data about restaurants. The following code
inserts data into the ``restaurants`` collection of the ``aggregation``
database:

.. literalinclude:: /code-snippets/aggregation/agg.js
:start-after: begin data insertion
:end-before: end data insertion
:language: javascript
:dedent:

.. tip::

For more information on connecting to your MongoDB deployment, see the :doc:`Connection Guide </connect>`.

Aggregation Example
~~~~~~~~~~~~~~~~~~~

To perform an aggregation, pass a list of aggregation stages to the
``collection.aggregate()`` method.

In the example, the aggregation pipeline uses the following aggregation stages:

- A :manual:`$match </reference/operator/aggregation/match/>` stage to filter for documents whose
``categories`` array field contains the element ``Bakery``.

- A :manual:`$group </reference/operator/aggregation/group/>` stage to group the matching documents by the ``stars``
field, accumulating a count of documents for each distinct value of ``stars``.

.. literalinclude:: /code-snippets/aggregation/agg.js
:start-after: begin aggregation
:end-before: end aggregation
:language: javascript
:dedent:

This example produces the following output:

.. code-block:: json
:copyable: false

{ _id: 4, count: 2 }
{ _id: 3, count: 1 }
{ _id: 5, count: 1 }

For more information, see the `aggregate() API documentation <{+api+}/classes/Collection.html#aggregate>`__.

.. _node-aggregation-tutorials-landing:
.. _node-aggregation-tutorials:

Aggregation Tutorials
---------------------

Aggregation tutorials provide detailed explanations of common
aggregation tasks in a step-by-step format. The tutorials are adapted
from examples in the `Practical MongoDB Aggregations book
<https://www.practical-mongodb-aggregations.com/>`__ by Paul Done.

Each tutorial includes the following sections:

- **Introduction**, which describes the purpose and common use cases of the
aggregation type. This section also describes the example and desired
outcome that the tutorial demonstrates.

- **Before You Get Started**, which describes the necessary databases,
collections, and sample data that you must have before building the
aggregation pipeline and performing the aggregation.

- **Tutorial**, which describes how to build and run the aggregation
pipeline. This section describes each stage of the completed
aggregation tutorial, and then explains how to run and interpret the
output of the aggregation.

At the end of each aggregation tutorial, you can find a link to a fully
runnable Node.js code file that you can run in your environment.

.. tip::

To learn more about performing aggregations, see the
:ref:`node-aggregation` guide.

.. _node-agg-tutorial-template-app:

Aggregation Template App
~~~~~~~~~~~~~~~~~~~~~~~~

Before you begin following an aggregation tutorial, you must set up a
new Node.js app. You can use this app to connect to a MongoDB
deployment, insert sample data into MongoDB, and run the aggregation
pipeline in each tutorial.

.. tip::

To learn how to install the driver and connect to MongoDB,
see the :ref:`node-get-started-download-and-install` and
:ref:`node-get-started-create-deployment` steps of the
Quick Start guide.

Once you install the driver, create a file called
``agg_tutorial.js``. Paste the following code in this file to create an
app template for the aggregation tutorials:

.. literalinclude:: /includes/aggregation/template-app.js
:language: javascript
:copyable: true

.. important::

In the preceding code, read the code comments to find the sections of
the code that you must modify for the tutorial you are following.

If you attempt to run the code without making any changes, you will
encounter a connection error.

For every tutorial, you must replace the connection string placeholder with
your deployment's connection string.

.. tip::

To learn how to locate your deployment's connection string, see the
:ref:`node-get-started-connection-string` step of the Quick Start guide.

For example, if your connection string is
``"mongodb+srv://mongodb-example:27017"``, your connection string assignment resembles
the following:

.. code-block:: javascript
:copyable: false

const uri = "mongodb+srv://mongodb-example:27017";

To run the completed file after you modify the template for a
tutorial, run the following command in your shell:

.. code-block:: bash

node agg_tutorial.js

Available Tutorials
~~~~~~~~~~~~~~~~~~~

- :ref:`node-aggregation-filtered-subset`
- :ref:`node-aggregation-group-total`
- :ref:`node-aggregation-arrays`
- :ref:`node-aggregation-one-to-one`
- :ref:`node-aggregation-multi-field`
- Pipeline stages have a memory limit of 100 megabytes by default. If required,
you can exceed this limit by enabling the `AllowDiskUse
<https://mongodb.github.io/node-mongodb-native/6.17/interfaces/AggregateOptions.html#allowDiskUse>`__
property of the ``AggregateOptions`` object that you pass to the
``Aggregate()`` method.

Additional Examples
~~~~~~~~~~~~~~~~~~~
Additional information
----------------------

To view step-by-step explanations of common aggregation tasks, see the
:ref:`node-aggregation-tutorials-landing`.
To view a full list of expression operators, see
:manual:`Aggregation Operators </reference/operator/aggregation/>`.

You can find another aggregation pipeline example in the `Aggregation
Framework with Node.js Tutorial
<https://www.mongodb.com/blog/post/quick-start-nodejs--mongodb--how-to-analyze-data-using-the-aggregation-framework>`_
blog post on the MongoDB website.
To learn about explaining MongoDB aggregation operations, see
:manual:`Explain Results </reference/explain-results/>` and
:manual:`Query Plans </core/query-plans/>`.
Loading
Loading