RFC: Auto Mask Secrets

[jasonwadsworth]

Description of the feature request

Automatically mask secret or password values in extra data.

Problem statement

A best practice is to never log secrets or passwords. There are times when I have an object that might have one of these things in it. I'd like to be able to pass in the object without needing to pull out the secrets manually.

Summary of the feature

If turned on, any extra data passed into the logger would be looked at and secrets/passwords would be masked. Consider the following data.

{
  username: "my-username",
  password: "my-password"
}

The code would see a key that contains "password" (this could be a configurable regex with a default) and would replace it with something like "*****", resulting in the following extra data being logged.

{
  username: "mu-username",
  password: "*****"
}

No secrets in logs

Code examples

const logger = new Logger();

const objectWithASecret = {
  username: "my-username",
  password: "my-password"
};

logger.debug('Logging in', { objectWithASecret });  

Benefits for you and the wider AWS community

Safer logging.

Describe alternatives you've considered

Creating a function that does this for me. The downside to this is twofold. One, I have to include the function call everywhere, so if I forget it I risk writing something I shouldn't to the logs. Two, the devex is slightly worse because I have to do something like { extraData: cleanExtraData(extraData) } instead of just { extraData }

Additional context

Related issues, RFCs

[willfarrell]

I agree that credentials should not be logged as per ASVS 7.1.1. There are other types of data that should not be logged as well based on who owns the data or where it is stored.

How would someone automate this?
a) Have a list of keys that should be hidden? English speaking developers might use password or token, but what about in other languages?
b) Detect secret like patterns? Secrets could look like other data?
c) Mix of both?

What are the performance implications need to be considered compared to other approaches?

In @middy/input-output-logger we opted for allowing a developer to list what paths to omit using omitPaths. In my experience secrets/PII is well known during development and not overwhelming to manually list per handler.

I believe GitHub Actions has this functionality built in, they might have a package that could be used. Git secrets scanner may also be a source of ideas for this (https://github.com/awslabs/git-secrets).

[dreamorosi]

Hi @jasonwadsworth thanks a lot for the feature request.

We have had a similar feature on the radar for all the runtimes, although it seems this one has a more reduced scope. You can find more details in this other issue aws-powertools/powertools-lambda-python#1173

I'm 100% on board and in favor of the concept of not logging secrets and leaving the responsibility to the developer seems like a sensible choice.

At the moment though, we want to make sure to make the base feature set of the core utilities as stable and mature as possible before considering new features. After going GA and having collected enough feedback from customers we'll definitely consider this type of request.

[jasonwadsworth]

I searched for something like this and didn't find it. Thanks for the link to the roadmap. Also completely understand on timing. This was just something that came up today so I thought I'd add it.

This is one case where I'd like to make it easier for devs to not make mistakes, so it would be nice to not leave it to the devs. Having it at the configuration point of creating the logger would be great, since that can be done in a single place.

[damianesteban]

I like this feature. I suggest that the user can define which keys to be filtered instead of having defaults.

[RajivSah]

+1