Skip to content

Not working as expected in rainy days #2237

Closed
@idaamit

Description

@idaamit

On redis connection failure, I do not get exception.
I have upgraded to redis 4.2.0 with the following lambda :

export const handler = async (
    event: APIGatewayEvent, context: Context, callback: APIGatewayProxyCallback
) => {
    try {
        console.log(`Lambda triggered with event: ${JSON.stringify(event)}`);
        context.callbackWaitsForEmptyEventLoop = false;
        const redisClient = redis.createClient({
            socket: {
                host: '127.0.0.1',
                port: 6379,
                reconnectStrategy: function (retriesAttemptedSoFar: number): number | Error {
                    if (retriesAttemptedSoFar > 3) {
                        // End reconnecting with error
                        return new Error(`Redis retry attempt=${retriesAttemptedSoFar}`);
                    }
                    console.info(` try reconnect after ${Math.min(retriesAttemptedSoFar * 10, 30)}ms`);
                    // reconnect after
                    return Math.min(retriesAttemptedSoFar * 10, 30);
                }
            },
        });
        redisClient.on('ready', () => {
            console.log(`Got "ready" event from Redis. Connection established.`);
        });
        redisClient.on('connect', () => {
            console.log(`Got "connect" event from Redis. Stream is connected to the server`);
        });
        redisClient.on('error', (err) => {
            console.log(`Got an ERROR from Redis. Details: ${err.message}`);
        });
        await redisClient.connect();
        let serverStates : string[] = [];
        const MAX_RETRIES = 3;
        let i = 0;
        while (i < MAX_RETRIES) {
            try {
                serverStates = await redisClient.lRange('123', 0, -1);
            } catch (e) {
                i++;
                const logMessage = i < MAX_RETRIES ?
                    `attempt ${i}: Failed to store message. Trying again...` :
                    `attempt ${i}: Failed to store message. All attempts failed`;
                console.warn(logMessage);
                await new Promise(resolve => setTimeout(resolve, 10));
            }
        }
        if (i >= MAX_RETRIES) {
            throw new Error('Exceeded Max Attempts to store message in Redis');
        }
        console.log(`ServerStates: ${JSON.stringify(serverStates)}`);

        console.log(`${JSON.stringify(event)} Lambda finished successfully`);
        callback(null, {
            statusCode: 200,
            body: 'theResult'
        });
    } catch (e) {
        const error = e as Error;
        console.error(`Lambda execution failed. Got error: ${error.message}`);
        callback(error);
    }
}
  • If I do not get exception from redis, all works.
  • If I do not specify reconnectStrategy and redis connection fail, the code trys to reconnect inifinitly.
  • If redis connection fail with reconnectStrategy code, lambda gets to the catch section after 3 re-connections as expected.

The problem is : If I succeed to connect to redis, but before lrage redis stops (redis connection fail Socket closed unexpectedly) redis client reconnects for 3 times as written in reconnectStrategy, and after the 3 re-connects, the code stops and do not get to the catch section, ending the lambda as successfully finished.

How can I get to the exception section in this scenario ?
In redis 3.1.X, in similar case lambda finished with exception.

Environment:

  • Node.js Version: v14.1.0
  • Redis Server Version: 6.2.6
  • Node Redis Version: 4.2.0
  • Platform: Windows 10

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions