Skip to content

PHPC-2366: Invoke drivers-evergreen-tools scripts with bash #1530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 6 additions & 68 deletions .evergreen/README.md
Original file line number Diff line number Diff line change
@@ -1,78 +1,16 @@
# Evergreen Notes
# Evergreen

See [Evergreen Build System Architecture](architecture.md) for more details.

## mongo-php-toolchain

[10gen/mongo-php-toolchain](https://github.com/10gen/mongo-php-toolchain) is
responsible for building PHP binaries that will be distributed to Evergreen
hosts.

The Debian and RHEL toolchain scripts are responsible for building individual
toolchains for each variant (e.g. PHP version, architecture). Debian and RHEL
are separate primarily for reasons related to OpenSSL.

## drivers-evergreen-tools

[mongodb-labs/drivers-evergreen-tools](https://github.com/mongodb-labs/drivers-evergreen-tools)
consists of common scripts used by many drivers (e.g. Mongo Orchestration). The
PHP driver clones this during the build process.

In some cases, we have modified scripts in the drivers-evergreen-tools
repository. For instance, the PHP driver uses its own `run-orchestration.sh`
script but still relies on the common `stop-orchestration.sh` script.

## PHP driver Evergreen configuration

`$PROJECT_DIRECTORY` is set by Evergreen and denotes the directory where the PHP
driver is built and tested (i.e. git checkout directory).

Various files have been copied and modified from drivers-evergreen-tools:

* `compile.sh`: This is the main entry point for building the driver and will
either call `compile-unix.sh` and `compile-windows.sh`; however, Windows
builds use a Cygwin environment and are not yet supported.
* `compile-unix.sh`: This includes architecture-specific flags and is used for
both Linux and macOS builds.
* `run-orchestration.sh`: This is very similar to the drivers-evergreen-tools
script but it is customized to use the PHP driver's own topology configs for
Mongo Orchestration.
* `start-orchestration.sh`: This is modified to install an older, tagged
version of Mongo Orchestration. This was necessary for a particular platform,
but may be something to look into in the future.

The PHP driver's `config.yml` is a very stripped down and heavily modified copy
of the drivers-evergreen-tools configuration. This file starts by defining
several functions, which consist of one or more commands (types supported by
Evergreen).

In the case of the `bootstrap mongo-orchestration` function, we execute our
modified `run-orchestration.sh` script and assign various environment variables
which have been defined by our matrix configuration (excluding the
`$PROJECT_DIRECTORY`, which is set by Evergreen itself).

In `run tests`, we also depend on environment variables (e.g. `$MONGODB_URI`)
that originates in `run-orchestration.sh` and are unpacked and assigned by the
`expansions.update` command within the `bootstrap mongo-orchestration` function.
Since Evergreen functions do not return values, assigning environment variables
is the preferred way to communicate down the line.

The `pre` and `post` block define a sequence of functions to run before and
after each build, respectively.

Build variants are the top-level scope in Evergreen. They consist of a list of
tasks (i.e. a square in Evergreen output) and a matrix. The matrix specification
is the combination of all included axes. For instance, the `tests-php5` matrix
may consist of all OS types with PHP5, MongoDB server version 4.0 (we need not
test all server versions), and lastly one or more specific PHP 5.x versions.
Such a matrix might yield a dozen variants, which will be run for each task in
the build variant configuration. The `display_name` option determines how the
build variant (combination of matrix and all tasks) is labeled in the Evergreen
UI.

The `axes` block defines a list of values for a specific group or ID, which is
used when defining a matrix. For instance, if a matrix includes 4 axes, it will
expand to all possible combinations of values within those axes. Care should be
taken to not create too many redundant combinations when defining matrices.

The `exclude_spec` field within a build variant allows us to exclude one or more
combinations from the generated matrix. We typically use this to exclude an
environment we know is not supported (e.g. MongoDB 3.0 and 3.2 on zSeries).
consists of common scripts used by many drivers (e.g. starting
[mongo-orchestration](https://github.com/10gen/mongo-orchestration)). The PHP
driver clones this during the build process.
2 changes: 1 addition & 1 deletion .evergreen/compile-unix.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -o errexit # Exit the script with error if any of the commands fail

# Find PHP binary path for the requested version
Expand Down
48 changes: 0 additions & 48 deletions .evergreen/compile-windows.sh

This file was deleted.

29 changes: 0 additions & 29 deletions .evergreen/compile.sh

This file was deleted.

15 changes: 8 additions & 7 deletions .evergreen/config/functions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ functions:
script: |
${PREPARE_SHELL}
SKIP_LEGACY_SHELL=true \
sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
bash ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
# run-orchestration generates expansion file with MONGODB_URI and CRYPT_SHARED_LIB_PATH
- command: expansions.update
params:
Expand All @@ -190,7 +190,7 @@ functions:
params:
script: |
${PREPARE_SHELL}
sh ${DRIVERS_TOOLS}/.evergreen/stop-orchestration.sh
bash ${DRIVERS_TOOLS}/.evergreen/stop-orchestration.sh

"run OCSP responder":
- command: shell.exec
Expand All @@ -202,7 +202,7 @@ functions:
shell: bash
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-ocsp-responder.sh
bash ${PROJECT_DIRECTORY}/.evergreen/run-ocsp-responder.sh

"run tests":
- command: shell.exec
Expand All @@ -221,7 +221,7 @@ functions:
working_dir: "src"
script: |
${PREPARE_SHELL}
sh ${PROJECT_DIRECTORY}/.evergreen/run-tests.sh
bash ${PROJECT_DIRECTORY}/.evergreen/run-tests.sh

"cleanup":
- command: shell.exec
Expand Down Expand Up @@ -254,13 +254,14 @@ functions:
${PREPARE_SHELL}
file="${DRIVERS_TOOLS}/.evergreen/install-dependencies.sh"
# Don't use ${file} syntax here because evergreen treats it as an empty expansion.
[ -f "$file" ] && sh $file || echo "$file not available, skipping"
[ -f "$file" ] && bash $file || echo "$file not available, skipping"

"start load balancer":
- command: shell.exec
params:
script: |
MONGODB_URI="${MONGODB_URI}" ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh start
MONGODB_URI="${MONGODB_URI}" \
bash ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh start
- command: expansions.update
params:
file: lb-expansion.yml
Expand All @@ -271,5 +272,5 @@ functions:
script: |
# Only run if a load balancer was started
if [ -n "${SINGLE_MONGOS_LB_URI}" ]; then
${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh stop
bash ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh stop
fi
2 changes: 1 addition & 1 deletion .evergreen/run-ocsp-responder.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -o errexit

# Run an mock OCSP responder server if necessary. This script should be invoked
Expand Down
2 changes: 1 addition & 1 deletion .evergreen/run-tests.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/sh
#!/usr/bin/env bash
set -o errexit # Exit the script with error if any of the commands fail

# Supported environment variables
Expand Down