From 5842f1f547eb3b6fe9bff5d9da944221180d4d65 Mon Sep 17 00:00:00 2001 From: Jeremy Mikola Date: Fri, 22 Mar 2024 09:44:17 -0400 Subject: [PATCH 1/3] Replace old Evergreen README content with link to architecture doc --- .evergreen/README.md | 74 ++++---------------------------------------- 1 file changed, 6 insertions(+), 68 deletions(-) diff --git a/.evergreen/README.md b/.evergreen/README.md index 58ea086bf..d5439e6a3 100644 --- a/.evergreen/README.md +++ b/.evergreen/README.md @@ -1,4 +1,6 @@ -# Evergreen Notes +# Evergreen + +See [Evergreen Build System Architecture](architecture.md) for more details. ## mongo-php-toolchain @@ -6,73 +8,9 @@ responsible for building PHP binaries that will be distributed to Evergreen hosts. -The Debian and RHEL toolchain scripts are responsible for building individual -toolchains for each variant (e.g. PHP version, architecture). Debian and RHEL -are separate primarily for reasons related to OpenSSL. - ## drivers-evergreen-tools [mongodb-labs/drivers-evergreen-tools](https://github.com/mongodb-labs/drivers-evergreen-tools) -consists of common scripts used by many drivers (e.g. Mongo Orchestration). The -PHP driver clones this during the build process. - -In some cases, we have modified scripts in the drivers-evergreen-tools -repository. For instance, the PHP driver uses its own `run-orchestration.sh` -script but still relies on the common `stop-orchestration.sh` script. - -## PHP driver Evergreen configuration - -`$PROJECT_DIRECTORY` is set by Evergreen and denotes the directory where the PHP -driver is built and tested (i.e. git checkout directory). - -Various files have been copied and modified from drivers-evergreen-tools: - - * `compile.sh`: This is the main entry point for building the driver and will - either call `compile-unix.sh` and `compile-windows.sh`; however, Windows - builds use a Cygwin environment and are not yet supported. - * `compile-unix.sh`: This includes architecture-specific flags and is used for - both Linux and macOS builds. - * `run-orchestration.sh`: This is very similar to the drivers-evergreen-tools - script but it is customized to use the PHP driver's own topology configs for - Mongo Orchestration. - * `start-orchestration.sh`: This is modified to install an older, tagged - version of Mongo Orchestration. This was necessary for a particular platform, - but may be something to look into in the future. - -The PHP driver's `config.yml` is a very stripped down and heavily modified copy -of the drivers-evergreen-tools configuration. This file starts by defining -several functions, which consist of one or more commands (types supported by -Evergreen). - -In the case of the `bootstrap mongo-orchestration` function, we execute our -modified `run-orchestration.sh` script and assign various environment variables -which have been defined by our matrix configuration (excluding the -`$PROJECT_DIRECTORY`, which is set by Evergreen itself). - -In `run tests`, we also depend on environment variables (e.g. `$MONGODB_URI`) -that originates in `run-orchestration.sh` and are unpacked and assigned by the -`expansions.update` command within the `bootstrap mongo-orchestration` function. -Since Evergreen functions do not return values, assigning environment variables -is the preferred way to communicate down the line. - -The `pre` and `post` block define a sequence of functions to run before and -after each build, respectively. - -Build variants are the top-level scope in Evergreen. They consist of a list of -tasks (i.e. a square in Evergreen output) and a matrix. The matrix specification -is the combination of all included axes. For instance, the `tests-php5` matrix -may consist of all OS types with PHP5, MongoDB server version 4.0 (we need not -test all server versions), and lastly one or more specific PHP 5.x versions. -Such a matrix might yield a dozen variants, which will be run for each task in -the build variant configuration. The `display_name` option determines how the -build variant (combination of matrix and all tasks) is labeled in the Evergreen -UI. - -The `axes` block defines a list of values for a specific group or ID, which is -used when defining a matrix. For instance, if a matrix includes 4 axes, it will -expand to all possible combinations of values within those axes. Care should be -taken to not create too many redundant combinations when defining matrices. - -The `exclude_spec` field within a build variant allows us to exclude one or more -combinations from the generated matrix. We typically use this to exclude an -environment we know is not supported (e.g. MongoDB 3.0 and 3.2 on zSeries). +consists of common scripts used by many drivers (e.g. starting +[mongo-orchestration](https://github.com/10gen/mongo-orchestration)). The PHP +driver clones this during the build process. From 23505edc69b3ae88f97336d23bcaed9b40f8b5ed Mon Sep 17 00:00:00 2001 From: Jeremy Mikola Date: Fri, 22 Mar 2024 10:13:27 -0400 Subject: [PATCH 2/3] PHPC-2366: Invoke drivers-evergreen-tools scripts with bash This also applies bash to PHPC's scripts since it makes no difference. --- .evergreen/compile-unix.sh | 2 +- .evergreen/config/functions.yml | 15 ++++++++------- .evergreen/run-ocsp-responder.sh | 2 +- .evergreen/run-tests.sh | 2 +- 4 files changed, 11 insertions(+), 10 deletions(-) diff --git a/.evergreen/compile-unix.sh b/.evergreen/compile-unix.sh index eaebc340a..353cecbfb 100755 --- a/.evergreen/compile-unix.sh +++ b/.evergreen/compile-unix.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -o errexit # Exit the script with error if any of the commands fail # Find PHP binary path for the requested version diff --git a/.evergreen/config/functions.yml b/.evergreen/config/functions.yml index 4628d8eaf..538abdea0 100644 --- a/.evergreen/config/functions.yml +++ b/.evergreen/config/functions.yml @@ -179,7 +179,7 @@ functions: script: | ${PREPARE_SHELL} SKIP_LEGACY_SHELL=true \ - sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh + bash ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh # run-orchestration generates expansion file with MONGODB_URI and CRYPT_SHARED_LIB_PATH - command: expansions.update params: @@ -190,7 +190,7 @@ functions: params: script: | ${PREPARE_SHELL} - sh ${DRIVERS_TOOLS}/.evergreen/stop-orchestration.sh + bash ${DRIVERS_TOOLS}/.evergreen/stop-orchestration.sh "run OCSP responder": - command: shell.exec @@ -202,7 +202,7 @@ functions: shell: bash script: | ${PREPARE_SHELL} - ${PROJECT_DIRECTORY}/.evergreen/run-ocsp-responder.sh + bash ${PROJECT_DIRECTORY}/.evergreen/run-ocsp-responder.sh "run tests": - command: shell.exec @@ -221,7 +221,7 @@ functions: working_dir: "src" script: | ${PREPARE_SHELL} - sh ${PROJECT_DIRECTORY}/.evergreen/run-tests.sh + bash ${PROJECT_DIRECTORY}/.evergreen/run-tests.sh "cleanup": - command: shell.exec @@ -254,13 +254,14 @@ functions: ${PREPARE_SHELL} file="${DRIVERS_TOOLS}/.evergreen/install-dependencies.sh" # Don't use ${file} syntax here because evergreen treats it as an empty expansion. - [ -f "$file" ] && sh $file || echo "$file not available, skipping" + [ -f "$file" ] && bash $file || echo "$file not available, skipping" "start load balancer": - command: shell.exec params: script: | - MONGODB_URI="${MONGODB_URI}" ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh start + MONGODB_URI="${MONGODB_URI}" \ + bash ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh start - command: expansions.update params: file: lb-expansion.yml @@ -271,5 +272,5 @@ functions: script: | # Only run if a load balancer was started if [ -n "${SINGLE_MONGOS_LB_URI}" ]; then - ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh stop + bash ${DRIVERS_TOOLS}/.evergreen/run-load-balancer.sh stop fi diff --git a/.evergreen/run-ocsp-responder.sh b/.evergreen/run-ocsp-responder.sh index e62f64268..376b37631 100755 --- a/.evergreen/run-ocsp-responder.sh +++ b/.evergreen/run-ocsp-responder.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -o errexit # Run an mock OCSP responder server if necessary. This script should be invoked diff --git a/.evergreen/run-tests.sh b/.evergreen/run-tests.sh index d46cf57ed..8a7524027 100755 --- a/.evergreen/run-tests.sh +++ b/.evergreen/run-tests.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/usr/bin/env bash set -o errexit # Exit the script with error if any of the commands fail # Supported environment variables From 0255d003221106b1eed6a830306372934309697a Mon Sep 17 00:00:00 2001 From: Jeremy Mikola Date: Fri, 22 Mar 2024 10:14:44 -0400 Subject: [PATCH 3/3] compile.sh and compile-windows.sh are obsolete --- .evergreen/compile-windows.sh | 48 ----------------------------------- .evergreen/compile.sh | 29 --------------------- 2 files changed, 77 deletions(-) delete mode 100755 .evergreen/compile-windows.sh delete mode 100755 .evergreen/compile.sh diff --git a/.evergreen/compile-windows.sh b/.evergreen/compile-windows.sh deleted file mode 100755 index 10d207276..000000000 --- a/.evergreen/compile-windows.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/bin/sh -set -o igncr # Ignore CR in this script -set -o errexit # Exit the script with error if any of the commands fail - -# Supported/used environment variables: -# CC Which compiler to use - - -case "$CC" in - # 64bit specific configuration - *Win64) - ;; - # 32bit specific configuration - *) - ;; -esac - -# Resolve the compiler name to correct MSBuild location -case "$CC" in - "Visual Studio 10 2010") - BUILD="/cygdrive/c/Windows/Microsoft.NET/Framework/v4.0.30319/MSBuild.exe" - ;; - "Visual Studio 10 2010 Win64") - BUILD="/cygdrive/c/Windows/Microsoft.NET/Framework64/v4.0.30319/MSBuild.exe" - ;; - "Visual Studio 12 2013") - BUILD="/cygdrive/c/Program Files (x86)/MSBuild/12.0/Bin/MSBuild.exe" - ;; - "Visual Studio 12 2013 Win64") - BUILD="/cygdrive/c/Program Files (x86)/MSBuild/12.0/Bin/MSBuild.exe" - ;; - "Visual Studio 14 2015") - BUILD="/cygdrive/c/Program Files (x86)/MSBuild/14.0/Bin/MSBuild.exe" - ;; - "Visual Studio 14 2015 Win64") - BUILD="/cygdrive/c/Program Files (x86)/MSBuild/14.0/Bin/MSBuild.exe" - ;; -esac - -export PATH=$PATH:`pwd`/tests:`pwd`/Debug:`pwd`/src/libbson/Debug -CMAKE="/cygdrive/c/cmake/bin/cmake" -INSTALL_DIR="C:/install-dir" - - -"$CMAKE" -G "$CC" "-DCMAKE_INSTALL_PREFIX=${INSTALL_DIR}" "-DBSON_ROOT_DIR=${INSTALL_DIR}" $CONFIGURE_FLAGS -"$BUILD" /m ALL_BUILD.vcxproj -"$BUILD" /m INSTALL.vcxproj - diff --git a/.evergreen/compile.sh b/.evergreen/compile.sh deleted file mode 100755 index 575ff1bbc..000000000 --- a/.evergreen/compile.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/sh -set -o errexit # Exit the script with error if any of the commands fail - - -DIR=$(dirname $0) - -OS=$(uname -s | tr '[:upper:]' '[:lower:]') -BUILDTOOL=${BUILDTOOL:-autotools} - -case "$OS" in - cygwin*) - sh $DIR/compile-windows.sh - ;; - - *) - # If compiling using multiple different build tools or variants - # that require wildly different scripting, - # this would be a good place to call the different scripts - case "$BUILDTOOL" in - cmake) - sh $DIR/compile-unix-cmake.sh - ;; - autotools) - sh $DIR/compile-unix.sh - ;; - esac - ;; -esac -