Skip to content

refactor(NODE-4685): unified spec runner to support logging tests #3578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
Mar 21, 2023

Conversation

W-A-James
Copy link
Contributor

@W-A-James W-A-James commented Feb 27, 2023

Description

Implement changes to unified spec runner to facilitate testing of logging spec implementation

What is changing?

  • Implement $$matchAsRoot operator
  • Implement $$matchAsDocument operator
  • Implement matching and interpretation of new properties
    • expectLogMessages
    • expectedLogMessages
    • expectedLogMessagesForClient
    • observeLogMessages
  • Implement unit tests for new operators
  • Stub out mechanism for UnifiedMongoClient to be able to collect logs from MongoLogger
Is there new documentation needed for these changes?

No

TODOs
  • Update AC of NODE-4849/file new ticket to include implementation of modifications to the unified spec runner that require the MongoLogger to be able to write logs to a Writable stream
  • Update AC of NODE-4849/file new ticket to implement total ordering for LogSeverity
  • Break out new ticket to add valid-pass and valid-fail tests to the unified-spec-format spec for $$matchAsRoot and $$matchAsDocument operators

Double check the following

  • Ran npm run check:lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@W-A-James W-A-James added the wip label Feb 27, 2023
@@ -169,6 +208,7 @@ export class UnifiedMongoClient extends MongoClient {
this.observedSdamEvents = (description.observeEvents ?? [])
.map(e => UnifiedMongoClient.SDAM_EVENT_NAME_LOOKUP[e])
.filter(e => !!e);
this.observedLogEvents = new Map();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will initialize this with the observedLogEvents passed in to the constructor in future, but need to implement NODE-4849 to have them take effect

@W-A-James W-A-James changed the title ci(NODE-4685): Implement test runner changes ci(NODE-4685): Implement changes to unified spec runner needed to interpret CLAM unified spec tests Mar 13, 2023
@@ -521,6 +521,9 @@ export function parseOptions(
const loggerFeatureFlag = Symbol.for('@@mdb.enableMongoLogger');
mongoOptions[loggerFeatureFlag] = mongoOptions[loggerFeatureFlag] ?? false;

const internalLoggerConfigSymbol = Symbol.for('@@mdb.internalLoggerConfig');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anywhere I should declare this symbol outside of this function? I didn't declare it anywhere else simply because it is something we only intend to use for testing.

Comment on lines 186 to 190
[Symbol.for('@@mdb.enableMongoLogger')]: true,
[Symbol.for('@@mdb.internalLoggerConfig')]: {
componentSeverities: description.observeLogMessages,
logDestination: logCollector
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing in the logDestination like this comes with the bonus that TS doesn't complain about options that don't exist


function getClient(address) {
return new MongoClient(`mongodb://${address}`, getEnvironmentalOptions());
}

// TODO(NODE-4813): Remove this class in favour of a simple object with a write method
/* TODO(NODE-4813): Ensure that the object that we replace this with has logic to convert the
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this TODO instead of adding the logic here since we are still solidifying what the shape of the logs will be

@W-A-James W-A-James removed the wip label Mar 20, 2023
@W-A-James W-A-James requested review from dariakp and nbbeeken March 20, 2023 18:13
@W-A-James W-A-James added the wip label Mar 20, 2023
…f github.com:mongodb/node-mongodb-native into NODE-4685/Easier_debugging_with_standardized_logging
Comment on lines 186 to 203
const camelToUpperSnake = (s: string) => {
const output: string[] = [];
for (const c of s) {
if (/[A-Z]/.test(c)) {
output.push('_');
output.push(c);
} else {
output.push(c);
}
}
return output.join('').toUpperCase();
};

// NOTE: this is done to override the logger environment variables
for (const key in description.observeLogMessages) {
componentSeverities['MONGODB_LOG_' + camelToUpperSnake(key)] =
description.observeLogMessages[key];
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is safe as we know that all the keys in description.observeLogMessages are the same as the spec-described loggable components and we want to be able to override our environment variables which are described in the spec to be in upper snake case (UPPER_SNAKE_CASE) with MONGODB_LOG_ as a prefix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would hard code the mapping given its a know limited set of keys and then we can find out if there are ones we are unaware of during test sync. But I don't feel strongly

@W-A-James W-A-James removed the wip label Mar 20, 2023
@W-A-James W-A-James requested a review from nbbeeken March 20, 2023 20:10
nbbeeken
nbbeeken previously approved these changes Mar 20, 2023
Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work, lgtm!

Comment on lines 186 to 203
const camelToUpperSnake = (s: string) => {
const output: string[] = [];
for (const c of s) {
if (/[A-Z]/.test(c)) {
output.push('_');
output.push(c);
} else {
output.push(c);
}
}
return output.join('').toUpperCase();
};

// NOTE: this is done to override the logger environment variables
for (const key in description.observeLogMessages) {
componentSeverities['MONGODB_LOG_' + camelToUpperSnake(key)] =
description.observeLogMessages[key];
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would hard code the mapping given its a know limited set of keys and then we can find out if there are ones we are unaware of during test sync. But I don't feel strongly

dariakp
dariakp previously approved these changes Mar 20, 2023
Copy link
Contributor

@dariakp dariakp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice work

Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just waiting on ci

@nbbeeken nbbeeken merged commit cb1ea8a into main Mar 21, 2023
@nbbeeken nbbeeken deleted the NODE-4685/Easier_debugging_with_standardized_logging branch March 21, 2023 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team Review Needs review from team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants