Feature: option to add prefix to all log messages

[zirkelc]

Hi,

it's me again 👋 😄

I would like to suggest an option to configure an optional prefix which is automatically being added to all logged messages. I have a Lambda that processes items from a database in sequence and logs various information to the console. At the moment I have a local prefix field that I insert into every log statement.

Pseudocode:

const users = await getUsers();

logger.info(`Checking ${users.length}` users`);

for(const user of users) {
  const prefix = `[${user.id}]`;

  // check 1...
  logger.info(`${prefix}: User ...`);

  // check 2...
  logger.info(`${prefix}: User ...`);

  // check 3...
  logger.info(`${prefix}: User ...`);
}

Why not use logger.appendKeys()?
The AWS management console CloudWatch search when used with a keywords only returns the rows having this keyword (e.g. the user id), but there is no option to show the rows before or after the match. It misses something like grep -A -B -C. So I often resort to show all logs and use the browser search on CMD+F, but this only work on visible text and not the hidden JSON that is only available on expanding a row.

Suggestion
The new prefix option should be available on the constructor and on logger.createChild().

const logger = new Logger();

for(const user of users) {
  const usersLogger = logger.createChild({ prefix: `[${user.id}]: ` });

  usersLogger.info(`User ...`);
  usersLogger.info(`User ...`);
  usersLogger.info(`User ...`);
}

[dreamorosi]

Hi @zirkelc, nice to see you here!

I am not sure I understand the issue with CloudWatch and appendKeys(). For example, using this code:

import { Logger } from '@aws-lambda-powertools/logger';

interface CustomEvent {
  userId?: string;
  text?: string;
}

const logger = new Logger({
  logLevel: 'DEBUG',
  serviceName: 'my-service',
});

export const handler = async (event: CustomEvent) => {
  logger.appendKeys({
    userId: event.userId || 'anonymous',
  });

  logger.debug('Handler invoked');
  logger.debug('Handler invoked 2');
  logger.debug('Handler invoked 3');

  return {
    statusCode: 200,
    body: JSON.stringify('Hello, World!'),
  };
};

and calling with payloads like userId123 and userId543 I was able to query all the logs for that entire request using the userId key as correlation ID, both for all requests:

and a specific userId:

Regarding the prefix feature, this looks like something very use-case specific and so I would suggest to look at using a custom log formatter that you can pass when calling the createChild() method which achieves the same thing you were looking for:

import { Logger, LogFormatter, LogItem } from '@aws-lambda-powertools/logger';
import type {
  LogAttributes,
  UnformattedAttributes,
} from '@aws-lambda-powertools/logger/types';

class PrefixFormatter extends LogFormatter {
  readonly #prefix: string;

  public constructor(prefix: string) {
    super();
    this.#prefix = prefix;
  }

  public formatAttributes(
    attributes: UnformattedAttributes,
    additionalLogAttributes: LogAttributes
  ): LogItem {
    return new LogItem({
      attributes: {
        ...attributes,
        message: `${this.#prefix}${attributes.message}`,
      },
    }).addAttributes(additionalLogAttributes);
  }
}

const logger = new Logger({
  logLevel: 'INFO',
});

export const handler = async () => {
  const prefixLogger = logger.createChild({
    logFormatter: new PrefixFormatter('foo '),
  });
  prefixLogger.info('Hello world');
};

Overall with this being an opinionated JSON-structured logger having a prefix to filter/query messages sounds like an anti-pattern - unless I am completely misunderstanding the request.

[zirkelc]

Hey @dreamorosi

thanks for the quick reply! I usually look into the CloudWatch logs in the management console or streamed into CLI/VSCode, or use Live Tail. You're right about CloudWatch Insights and using a key to filter the records. IMO the downside of Log Insights is that you have to be certain you know what you're searching for, otherwise you easily miss something because you don't see the context, i.e. the log lines before/after the the search result. For example, if the Lambda ends with an unhandled error, it wouldn't appear in the log results. That's the reason I usually resort to using the default Log Group interface to see the request start to end.

Overall you're right, this is very use case specific and not necessarily a best practice for a structured logger. Your suggested solution with a custom formatter is a good alternative!

Thanks!