Skip to content

fix: update server description when equal #2256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 50 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
d4a51a4
doc: remove needless reference to current driver version
mbroadst Jan 24, 2020
41ee8e0
docs: provide basic migration updates for v3.5.xwq
mbroadst Jan 24, 2020
ffe92a9
Merge branch '3.5'
mbroadst Jan 29, 2020
2030a69
chore: sync retryable-writes tests
mbroadst Jan 29, 2020
5f04660
test: update retryable-writes test runner to use new format
mbroadst Jan 29, 2020
7bbad9c
refactor: make `errorLabels` a private set of `MongoError`
mbroadst Feb 1, 2020
c775a4a
feat: expand use of error labels for retryable writes
mbroadst Feb 2, 2020
bb4689e
chore: resync retryable-writes tests for 4.3 mongos issues
mbroadst Feb 5, 2020
9b80c24
refactor: explicitly return a boolean for errorLabel presence
mbroadst Feb 5, 2020
6449f04
test: all tests simulating stepdown must now include an error code
mbroadst Feb 5, 2020
c1bff29
test: force use of a single host for sharded topology tests
mbroadst Feb 5, 2020
a89d491
test: ensure OP_MSG exhaust functional test is run against single
mbroadst Feb 5, 2020
79f4c65
test: remove assertion that `mongodb-core` exists in metadata
mbroadst Feb 5, 2020
fefc165
feat: use error labels for retryable writes in legacy topologies
mbroadst Feb 5, 2020
265fe40
fix: only consider MongoError subclasses for retryability
mbroadst Feb 5, 2020
68170da
Merge branch '3.5'
mbroadst Feb 6, 2020
6b9ff05
feat: support shorter SCRAM conversations
mbroadst Feb 10, 2020
292fe08
chore: sync CRUD v2 spec tests for 4.4 compat
mbroadst Feb 10, 2020
faee15b
feat: support passing a hint to findOneAndReplace/findOneAndUpdate
mbroadst Feb 11, 2020
dbc0b37
feat: support `allowDiskUse` for find commands
mbroadst Feb 11, 2020
a110ee4
chore: sync read-write-concern spec tests
mbroadst Feb 11, 2020
e855c83
fix: pass options into `commandSupportsReadConcern`
mbroadst Feb 11, 2020
c1ed2c1
fix: use properly camel cased form of `mapReduce` for command
mbroadst Feb 11, 2020
637f428
fix: support write concern provided as string in `fromOptions`
mbroadst Feb 11, 2020
62b39a0
refactor: begin to provide formal specs for test operations
mbroadst Feb 11, 2020
11f8792
test: improve testing of default write concern
mbroadst Feb 11, 2020
8b8a20c
refactor: spec test runner should support pre-sessions mongodb
mbroadst Feb 11, 2020
24155e7
feat: deprecate `oplogReplay` for find commands
mbroadst Feb 13, 2020
6d3f313
feat: bump wire protocol version for 4.4
mbroadst Feb 13, 2020
979d41e
fix: store name of collection for more informative error messages
mbroadst Feb 13, 2020
acdb648
doc: document new `background` option for `validateCollection`
mbroadst Feb 13, 2020
db991d6
fix(sdam): use ObjectId comparison to track maxElectionId
mbroadst Feb 21, 2020
b98a00a
chore: move sdam spec unit tests from `unit/core` to `unit/sdam`
mbroadst Feb 21, 2020
7f3cfba
feat: add MONGODB-AWS as a supported auth mechanism
mbroadst Feb 13, 2020
2d607fa
refactor: don't authenticate monitoring connections
mbroadst Feb 26, 2020
c4add7b
Merge branch '3.5'
mbroadst Feb 26, 2020
e6dc1f4
fix: don't depend on private node api for `Timeout` wrapper
mbroadst Feb 26, 2020
69d10ba
docs: fix typo and formatting of bulletted lists
stevenhair Feb 26, 2020
9ad0def
docs: update community links to use MongoDB Community forums
sindbach Mar 3, 2020
ade9a62
refactor: default to unified topology
mbroadst Feb 28, 2020
6aa2434
feat: remove legacy topology types
mbroadst Mar 2, 2020
79cbe09
chore: remove `shallowClone` util helper
mbroadst Mar 5, 2020
8c5197a
refactor: merge core and native error files
mbroadst Mar 6, 2020
8807508
refactor: remove the core module
mbroadst Mar 8, 2020
4c852e7
chore: remove evergreen configuration for unified topology
mbroadst Mar 9, 2020
2355997
doc(examples): remove unused imported module `assert`
isaacdecoded Mar 9, 2020
8a79aad
fix: update server description when equal to keep client-tracked attr…
emadum Mar 9, 2020
e8c4561
clean up
emadum Mar 9, 2020
bbacab4
add spec test sdam/rs/repeated
emadum Mar 10, 2020
223f99b
Merge branch '3.5' into NODE-2474/update-equal-server-description
emadum Mar 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
950 changes: 323 additions & 627 deletions .evergreen/config.yml

Large diffs are not rendered by default.

178 changes: 176 additions & 2 deletions .evergreen/config.yml.in
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ functions:
# If this was a patch build, doing a fresh clone would not actually test the patch
cp -R ${PROJECT_DIRECTORY}/ $DRIVERS_TOOLS
else
git clone git://github.com/mongodb-labs/drivers-evergreen-tools.git $DRIVERS_TOOLS
# git clone git://github.com/mongodb-labs/drivers-evergreen-tools.git $DRIVERS_TOOLS
git clone git://github.com/bazile-clyde/drivers-evergreen-tools.git $DRIVERS_TOOLS
fi
echo "{ \"releases\": { \"default\": \"$MONGODB_BINARIES\" }}" > $MONGO_ORCHESTRATION_HOME/orchestration.config

Expand All @@ -94,7 +95,7 @@ functions:
params:
script: |
${PREPARE_SHELL}
MONGODB_VERSION=${VERSION} TOPOLOGY=${TOPOLOGY} bash ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
MONGODB_VERSION=${VERSION} TOPOLOGY=${TOPOLOGY} ORCHESTRATION_FILE=${ORCHESTRATION_FILE} bash ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
# run-orchestration generates expansion file with the MONGODB_URI for the cluster
- command: expansions.update
params:
Expand Down Expand Up @@ -186,6 +187,179 @@ functions:
# DO NOT ECHO WITH XTRACE (which PREPARE_SHELL does)
NODE_LTS_NAME='${NODE_LTS_NAME}' ATLAS_REPL='${atlas_repl}' ATLAS_SHRD='${atlas_shrd}' ATLAS_FREE='${atlas_free}' ATLAS_TLS11='${atlas_tls11}' ATLAS_TLS12='${atlas_tls12}' bash ${PROJECT_DIRECTORY}/.evergreen/run-atlas-tests.sh

"add aws auth variables to file":
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
cat <<EOF > ${DRIVERS_TOOLS}/.evergreen/auth_aws/aws_e2e_setup.json
{
"iam_auth_ecs_account" : "${iam_auth_ecs_account}",
"iam_auth_ecs_secret_access_key" : "${iam_auth_ecs_secret_access_key}",
"iam_auth_ecs_account_arn": "arn:aws:iam::557821124784:user/authtest_fargate_user",
"iam_auth_ecs_cluster": "${iam_auth_ecs_cluster}",
"iam_auth_ecs_task_definition": "${iam_auth_ecs_task_definition}",
"iam_auth_ecs_subnet_a": "${iam_auth_ecs_subnet_a}",
"iam_auth_ecs_subnet_b": "${iam_auth_ecs_subnet_b}",
"iam_auth_ecs_security_group": "${iam_auth_ecs_security_group}",
"iam_auth_assume_aws_account" : "${iam_auth_assume_aws_account}",
"iam_auth_assume_aws_secret_access_key" : "${iam_auth_assume_aws_secret_access_key}",
"iam_auth_assume_role_name" : "${iam_auth_assume_role_name}",
"iam_auth_ec2_instance_account" : "${iam_auth_ec2_instance_account}",
"iam_auth_ec2_instance_secret_access_key" : "${iam_auth_ec2_instance_secret_access_key}",
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}"
}
EOF

run aws auth test with regular aws credentials:
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
${MONGODB_BINARIES}/mongo aws_e2e_regular_aws.js
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
alias urlencode='python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])"'
USER=$(urlencode ${iam_auth_ecs_account})
PASS=$(urlencode ${iam_auth_ecs_secret_access_key})
export MONGODB_URI="mongodb://$USER:$PASS@localhost:27017/aws?authMechanism=MONGODB-AWS"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-mongodb-aws-test.sh

run aws auth test with assume role credentials:
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
${MONGODB_BINARIES}/mongo aws_e2e_assume_role.js
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
alias urlencode='python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])"'
USER=$(jq -r '.AccessKeyId' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
USER=$(urlencode $USER)
PASS=$(jq -r '.SecretAccessKey' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
PASS=$(urlencode $PASS)
SESSION_TOKEN=$(jq -r '.SessionToken' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
SESSION_TOKEN=$(urlencode $SESSION_TOKEN)
export MONGODB_URI="mongodb://$USER:$PASS@localhost:27017/aws?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:$SESSION_TOKEN"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-mongodb-aws-test.sh

run aws auth test with aws EC2 credentials:
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
${MONGODB_BINARIES}/mongo aws_e2e_ec2.js
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-mongodb-aws-test.sh

run aws auth test with aws credentials as environment variables:
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
cat <<EOF > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ACCESS_KEY_ID=${iam_auth_ecs_account}
export AWS_SECRET_ACCESS_KEY=${iam_auth_ecs_secret_access_key}
export MONGODB_URI="mongodb://localhost:27017/aws?authMechanism=MONGODB-AWS"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-mongodb-aws-test.sh

run aws auth test with aws credentials and session token as environment variables:
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ACCESS_KEY_ID=$(jq -r '.AccessKeyId' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
export AWS_SECRET_ACCESS_KEY=$(jq -r '.SecretAccessKey' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
export AWS_SESSION_TOKEN=$(jq -r '.SessionToken' ${DRIVERS_TOOLS}/.evergreen/auth_aws/creds.json)
export MONGODB_URI="mongodb://localhost:27017/aws?authMechanism=MONGODB-AWS"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
${PROJECT_DIRECTORY}/.evergreen/run-mongodb-aws-test.sh

run aws ECS auth test:
- command: shell.exec
type: test
params:
working_dir: src
script: |
${PREPARE_SHELL}
AUTH_AWS_DIR=${DRIVERS_TOOLS}/.evergreen/auth_aws
ECS_SRC_DIR=$AUTH_AWS_DIR/src

# fix issue with `TestData` in SERVER-46340
sed -i '1s+^+TestData = {};\n+' $AUTH_AWS_DIR/lib/ecs_hosted_test.js

# pack up project directory to ssh it to the container
mkdir -p $ECS_SRC_DIR/.evergreen
cp $PROJECT_DIRECTORY/.evergreen/run-mongodb-aws-ecs-test.sh $ECS_SRC_DIR/.evergreen
tar -czf $ECS_SRC_DIR/src.tgz -C $PROJECT_DIRECTORY .

cd $AUTH_AWS_DIR
cat <<EOF > setup.js
const mongo_binaries = "$MONGODB_BINARIES";
const project_dir = "$ECS_SRC_DIR";
EOF

cat setup.js
mongo --nodb setup.js aws_e2e_ecs.js

pre:
- func: "fetch source"
- func: "prepare resources"
Expand Down
83 changes: 38 additions & 45 deletions .evergreen/generate_evergreen_tasks.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,43 +6,16 @@ const yaml = require('js-yaml');

const LATEST_EFFECTIVE_VERSION = '5.0';
const MONGODB_VERSIONS = ['latest', '4.2', '4.0', '3.6', '3.4', '3.2', '3.0', '2.6'];
const NODE_VERSIONS = ['dubnium', 'carbon', 'boron', 'argon'];
const TOPOLOGIES = ['server', 'replica_set', 'sharded_cluster'].concat([
'server-unified',
'replica_set-unified',
'sharded_cluster-unified'
]);

const NODE_VERSIONS = ['carbon', 'dubnium', 'erbium'];
const TOPOLOGIES = ['server', 'replica_set', 'sharded_cluster'];
const OPERATING_SYSTEMS = [
// Debian
{
name: 'debian71-test',
display_name: 'Debian 7.1',
run_on: 'debian71-test',
mongoVersion: '<4.0',
nodeVersions: ['argon', 'boron']
},
{
name: 'debian81-test',
display_name: 'Debian 8.1',
run_on: 'debian81-test',
mongoVersion: '>=3.4 <4.2'
},
// TODO: once we know how to test debian 9.x
// {
// name: 'debian91-test',
// display_name: 'Debian 9.1',
// run_on: 'debian91-test',
// mongoVersion: '>=4.0'
// },
// Amazon Linux
{
name: 'linux-64-amzn-test',
display_name: 'Amazon Linux (Enterprise)',
run_on: 'linux-64-amzn-test',
mongoVersion: '<4.0',
nodeVersions: ['argon', 'boron']
},
// macos
{
name: 'macos-1014',
Expand Down Expand Up @@ -81,8 +54,7 @@ const OPERATING_SYSTEMS = [
display_name: 'Ubuntu 16.04',
run_on: 'ubuntu1604-test',
mongoVersion: '>=3.2',
clientEncryption: true,
nodeVersions: ['dubnium', 'carbon', 'boron']
clientEncryption: true
},
{
name: 'ubuntu1604-arm64-small',
Expand All @@ -102,15 +74,6 @@ const OPERATING_SYSTEMS = [
run_on: 'ubuntu1804-arm64-test',
mongoVersion: '>=4.2'
}

// reenable when these are actually running 7.2, or we release a 7.4 rpm
// {
// name: 'rhel72-zseries-test',
// display_name: 'RHEL 7.2 (zSeries)',
// run_on: 'rhel72-zseries-test',
// mongoVersion: '>=3.4'
// },

// Windows. reenable this when nvm supports windows, or we settle on an alternative tool
// {
// name: 'windows-64-vs2010-test',
Expand All @@ -131,7 +94,7 @@ const OPERATING_SYSTEMS = [
Object.assign(
{
mongoVersion: '>=2.6',
nodeVersion: 'argon',
nodeVersion: 'carbon',
auth: false
},
osConfig
Expand All @@ -143,10 +106,6 @@ const TASKS = [];
function makeTask({ mongoVersion, topology }) {
let topologyForTest = topology;
let runTestsCommand = { func: 'run tests' };
if (topology.indexOf('-unified') !== -1) {
topologyForTest = topology.split('-unified')[0];
runTestsCommand = { func: 'run tests', vars: { UNIFIED: 1 } };
}

return {
name: `test-${mongoVersion}-${topology}`,
Expand Down Expand Up @@ -189,6 +148,29 @@ TASKS.push({
]
});

TASKS.push({
name: 'aws-auth-test',
commands: [
{ func: 'install dependencies' },
{
func: 'bootstrap mongo-orchestration',
vars: {
AUTH: 'auth',
ORCHESTRATION_FILE: 'auth-aws.json',
TOPOLOGY: 'server'
},
},
{ func: 'add aws auth variables to file' },
{ func: 'run aws auth test with regular aws credentials' },
{ func: 'run aws auth test with assume role credentials' },
{ func: 'run aws auth test with aws EC2 credentials' },
{ func: 'run aws auth test with aws credentials as environment variables' },
{ func: 'run aws auth test with aws credentials and session token as environment variables' },
{ func: 'run aws ECS auth test' }
]
});


const BUILD_VARIANTS = [];

const getTaskList = (() => {
Expand Down Expand Up @@ -242,6 +224,17 @@ OPERATING_SYSTEMS.forEach(
}
);

// special case for MONGODB-AWS authentication
BUILD_VARIANTS.push({
name: 'ubuntu1804-test-mongodb-aws',
display_name: 'MONGODB-AWS Auth test',
run_on: 'ubuntu1804-test',
expansions: {
NODE_LTS_NAME: 'carbon'
},
tasks: ['aws-auth-test']
})

const fileData = yaml.safeLoad(fs.readFileSync(`${__dirname}/config.yml.in`, 'utf8'));

fileData.tasks = (fileData.tasks || []).concat(TASKS);
Expand Down
19 changes: 19 additions & 0 deletions .evergreen/run-mongodb-aws-ecs-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash
set -o xtrace # Write all commands first to stderr
set -o errexit # Exit the script with error if any of the commands fail

MONGODB_URI="$1"
PROJECT_DIRECTORY="$(pwd)/src"

# untar packed archive
cd $PROJECT_DIRECTORY
tar -xzf src.tgz .

# load node.js
set +x
export NVM_DIR="${PROJECT_DIRECTORY}/node-artifacts/nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
set -x

# run the tests
MONGODB_URI=$MONGODB_URI MONGODB_UNIFIED_TOPOLOGY=1 npx mocha test/functional/mongodb_aws.test.js
24 changes: 24 additions & 0 deletions .evergreen/run-mongodb-aws-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
# set -o xtrace # Write all commands first to stderr
set -o errexit # Exit the script with error if any of the commands fail

MONGODB_URI=${MONGODB_URI:-}

# ensure no secrets are printed in log files
set +x

# load node.js environment
export NVM_DIR="${PROJECT_DIRECTORY}/node-artifacts/nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

# the default connection string, may be overridden by the environment script
export MONGODB_URI="mongodb://localhost:27017/aws?authMechanism=MONGODB-AWS"

# load the script
shopt -s expand_aliases # needed for `urlencode` alias
[ -s "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh" ] && source "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh"

# revert to show test output
set -x

MONGODB_UNIFIED_TOPOLOGY=1 npx mocha test/functional/mongodb_aws.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ The MongoClient connection method returns a Promise if no callback is passed to

```js
const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');

(async function() {
// Connection URL
Expand Down
Loading