Skip to content

Avoid double processing of default cert bundle in AL2023 #1661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions LambdaRuntimeDockerfiles/Images/net8/amd64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ RUN dotnet build "Amazon.Lambda.RuntimeSupport.csproj" /p:ExecutableOutputType=t
FROM builder AS publish
RUN dotnet publish "Amazon.Lambda.RuntimeSupport.csproj" /p:ExecutableOutputType=true /p:GenerateDocumentationFile=false /p:TargetFrameworks=net8.0 -f net8.0 --runtime linux-x64 --self-contained false -p:PublishReadyToRun=true -c Release -o /app/publish
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix /app/publish/bootstrap.sh && \
mv /app/publish/bootstrap.sh /app/publish/bootstrap && \
RUN dos2unix /app/publish/bootstrap-al2023.sh && \
mv /app/publish/bootstrap-al2023.sh /app/publish/bootstrap && \
chmod +x /app/publish/bootstrap


Expand Down
4 changes: 2 additions & 2 deletions LambdaRuntimeDockerfiles/Images/net8/arm64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ RUN dotnet build "Amazon.Lambda.RuntimeSupport.csproj" /p:ExecutableOutputType=t
FROM builder AS publish
RUN dotnet publish "Amazon.Lambda.RuntimeSupport.csproj" /p:ExecutableOutputType=true /p:GenerateDocumentationFile=false /p:TargetFrameworks=net8.0 -f net8.0 --runtime linux-arm64 --self-contained false -p:PublishReadyToRun=true -c Release -o /app/publish
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix /app/publish/bootstrap.sh && \
mv /app/publish/bootstrap.sh /app/publish/bootstrap && \
RUN dos2unix /app/publish/bootstrap-al2023.sh && \
mv /app/publish/bootstrap-al2023.sh /app/publish/bootstrap && \
chmod +x /app/publish/bootstrap


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,56 @@ public async Task ExceptionTests(string handler, string input, string expectedEr
Assert.Equal(expectedErrorMessage, exception["errorMessage"].ToString());
}

private async Task UpdateHandlerAsync(string handler)
/// <summary>
/// This test is checking the logic added to the bootstrap.sh to change the SSL_CERT_FILE
/// environment variable for AL2023.
/// </summary>
/// <param name="envName"></param>
/// <param name="expectedValue"></param>
/// <param name="setValue"></param>
/// <returns></returns>
[Theory]
#if NET8_0_OR_GREATER
[InlineData("SSL_CERT_FILE", "\"/tmp/noop\"", null)]
[InlineData("SSL_CERT_FILE", "\"/tmp/my-bundle\"", "/tmp/my-bundle")]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This inline data does support that setting/getting env variable work, but I don't think it's important (here AFAIK).

Is there a way we can start a process via shell and set SSL_CERT_FILE=somevalue and ensure our shell script honor that. If we can't I would understand, but that's a value addition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This inline data does support that setting/getting env variable work, but I don't think it's important (here AFAIK).

I need to make sure if a user wants to override the SSL_CERT_FILE our setting of the env var doesn't override their value.

Is there a way we can start a process via shell and set SSL_CERT_FILE=somevalue and ensure our shell script honor that. If we can't I would understand, but that's a value addition.

Not sure what you mean. The shell script is being executed because it was built into the OCI image of the Lambda function. So it executed just like a normal Lambda invocation. I can't run the script locally for variety of reasons but the biggest one is the test is likely being run from a Windows dev environment.

Copy link

@birojnayak birojnayak Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean. The shell script is being executed because it was built into the OCI image of the Lambda function. So it executed just like a normal Lambda invocation. I can't run the script locally for variety of reasons but the biggest one is the test is likely being run from a Windows dev environment.

I meant to say if there is a way to test below

if [ -z "${SSL_CERT_FILE}"]; then
  export SSL_CERT_FILE="/tmp/noop"
fi

If it's running in Windows , we can't do much.

#else
[InlineData("SSL_CERT_FILE", "", null)]
[InlineData("SSL_CERT_FILE", "/tmp/my-bundle", "/tmp/my-bundle")]
#endif
public async Task CheckEnvironmentVariable(string envName, string expectedValue, string setValue)
{
var envVariables = new Dictionary<string, string>();
if(setValue != null)
{
envVariables[envName] = setValue;
}

await UpdateHandlerAsync("ImageFunction::ImageFunction.Function::GetEnvironmentVariable", envVariables);

var payload = JsonConvert.SerializeObject(envName);
var invokeResponse = await InvokeFunctionAsync(payload);

Assert.True(invokeResponse.HttpStatusCode == System.Net.HttpStatusCode.OK);
Assert.True(invokeResponse.FunctionError == null, "Failed invoke with error: " + invokeResponse.FunctionError);

await using var responseStream = invokeResponse.Payload;
var actualValue = new StreamReader(responseStream).ReadToEnd();

Assert.Equal(expectedValue, actualValue);
}

private async Task UpdateHandlerAsync(string handler, Dictionary<string, string> environmentVariables = null)
{
var updateFunctionConfigurationRequest = new UpdateFunctionConfigurationRequest
{
FunctionName = _functionName,
ImageConfig = new ImageConfig()
{
Command = {handler},
},
Environment = new Amazon.Lambda.Model.Environment
{
Variables = environmentVariables ?? new Dictionary<string, string>()
}
};
await _lambdaClient.UpdateFunctionConfigurationAsync(updateFunctionConfigurationRequest);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,11 @@ public string VerifyTzData(ILambdaContext lambdaContext)
return SuccessResult;
}

public string GetEnvironmentVariable(string name)
{
return Environment.GetEnvironmentVariable(name) ?? string.Empty;
}

#endregion

#region Private methods
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,8 @@
<None Update="bootstrap.sh">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<None Update="bootstrap-al2023.sh">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
132 changes: 132 additions & 0 deletions Libraries/src/Amazon.Lambda.RuntimeSupport/bootstrap-al2023.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
#!/bin/bash

# .NET on Linux uses OpenSSL to handle certificates. The .NET runtime will load the certs by first reading
# the default cert bundle file which can be overriden by the SSL_CERT_FILE env var. Then it will load the
# certs in the default cert directory which can be overriden by the SSL_CERT_DIR env var. On AL2023
# The default cert bundle file, via symbolic links, resolves to being in a file under the default cert directory.
# This means the default cert bundle file is double loaded causing a cold start performance hit. This logic
# sets the SSL_CERT_FILE to a noop file if SSL_CERT_FILE hasn't been explicitly
# set. This avoid the double load of the default cert bundle file.
if [ -z "${SSL_CERT_FILE}"]; then
export SSL_CERT_FILE="/tmp/noop"
fi

# This script is used to locate 2 files in the /var/task folder, where the end-user assembly is located
# The 2 files are <assembly name>.deps.json and <assembly name>.runtimeconfig.json
# These files are used to add the end-user assembly into context and make the code reachable to the dotnet process
# Since the file names are not known in advance, we use this shell script to find the files and pass them to the dotnet process as parameters
# You can improve cold-start performance by setting the LAMBDA_DOTNET_MAIN_ASSEMBLY environment variable and specifying the assembly name
# LAMBDA_TASK_ROOT is inherited from the Lambda execution environment/base image as "/var/task", but can be overridden for use in custom images.
if [ -z "${LAMBDA_TASK_ROOT}" ]; then
echo "Error: Environment variable LAMBDA_TASK_ROOT needs to be defined in order for the Lambda Runtime to load the function handler to be executed." 1>&2
exit 101
fi

if [ ! -d "${LAMBDA_TASK_ROOT}" ] | [ -z "$(ls -A ${LAMBDA_TASK_ROOT})" ]; then
echo "Error: .NET binaries for Lambda function are not correctly installed in the ${LAMBDA_TASK_ROOT} directory of the image when the image was built. The ${LAMBDA_TASK_ROOT} directory is missing." 1>&2
exit 102
fi

# Get version of Lambda .NET runtime if available
export "$(grep LAMBDA_RUNTIME_NAME /var/runtime/runtime-release 2>/dev/null || echo LAMBDA_RUNTIME_NAME=dotnet_custom)"
export AWS_EXECUTION_ENV="AWS_Lambda_${LAMBDA_RUNTIME_NAME}"

export DOTNET_ROOT="/var/lang/bin"
DOTNET_BIN="${DOTNET_ROOT}/dotnet"
DOTNET_EXEC="exec"
DOTNET_ARGS=()
EXECUTABLE_BINARY_EXIST=false

LAMBDA_HANDLER=""
# Command-line parameter has precedence over "_HANDLER" environment variable
if [ -n "${1}" ]; then
LAMBDA_HANDLER="${1}"
elif [ -n "${_HANDLER}" ]; then
LAMBDA_HANDLER="${_HANDLER}"
else
echo "Error: No Lambda Handler function was specified." 1>&2
exit 103
fi

HANDLER_COL_INDEX=$(expr index "${LAMBDA_HANDLER}" ":")

if [[ "${HANDLER_COL_INDEX}" == 0 ]]; then
EXECUTABLE_ASSEMBLY="${LAMBDA_TASK_ROOT}/${LAMBDA_HANDLER}"
EXECUTABLE_BINARY="${LAMBDA_TASK_ROOT}/${LAMBDA_HANDLER}"
if [[ "${EXECUTABLE_ASSEMBLY}" != *.dll ]]; then
EXECUTABLE_ASSEMBLY="${EXECUTABLE_ASSEMBLY}.dll"
fi
if [[ -f "${EXECUTABLE_ASSEMBLY}" ]]; then
DOTNET_ARGS+=("${EXECUTABLE_ASSEMBLY}")
elif [[ -f "${EXECUTABLE_BINARY}" ]]; then
EXECUTABLE_BINARY_EXIST=true
else
echo "Error: executable assembly ${EXECUTABLE_ASSEMBLY} or binary ${EXECUTABLE_BINARY} not found." 1>&2
exit 104
fi
else
if [ -n "${LAMBDA_DOTNET_MAIN_ASSEMBLY}" ]; then
if [[ "${LAMBDA_DOTNET_MAIN_ASSEMBLY}" == *.dll ]]; then
ASSEMBLY_NAME="${LAMBDA_DOTNET_MAIN_ASSEMBLY::-4}"
else
ASSEMBLY_NAME="${LAMBDA_DOTNET_MAIN_ASSEMBLY}"
fi
else
ASSEMBLY_NAME="${LAMBDA_HANDLER::${HANDLER_COL_INDEX}-1}"
fi

DEPS_FILE="${LAMBDA_TASK_ROOT}/${ASSEMBLY_NAME}.deps.json"
if ! [ -f "${DEPS_FILE}" ]; then
DEPS_FILES=( "${LAMBDA_TASK_ROOT}"/*.deps.json )

# Check if there were any matches to the *.deps.json glob, and that the glob was resolved
# This makes the matching independent from the global `nullopt` shopt's value (https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html)
if [ "${#DEPS_FILES[@]}" -ne 1 ] || echo "${DEPS_FILES[0]}" | grep -q -F '*'; then
echo "Error: .NET binaries for Lambda function are not correctly installed in the ${LAMBDA_TASK_ROOT} directory of the image when the image was built. The ${LAMBDA_TASK_ROOT} directory is missing the required .deps.json file." 1>&2
exit 105
fi
DEPS_FILE="${DEPS_FILES[0]}"
fi

RUNTIMECONFIG_FILE="${LAMBDA_TASK_ROOT}/${ASSEMBLY_NAME}.runtimeconfig.json"
if ! [ -f "${RUNTIMECONFIG_FILE}" ]; then
RUNTIMECONFIG_FILES=( "${LAMBDA_TASK_ROOT}"/*.runtimeconfig.json )

# Check if there were any matches to the *.runtimeconfig.json glob, and that the glob was resolved
# This makes the matching independent from the global `nullopt` shopt's value (https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html)
if [ "${#RUNTIMECONFIG_FILES[@]}" -ne 1 ] || echo "${RUNTIMECONFIG_FILES[0]}" | grep -q -F '*'; then
echo "Error: .NET binaries for Lambda function are not correctly installed in the ${LAMBDA_TASK_ROOT} directory of the image when the image was built. The ${LAMBDA_TASK_ROOT} directory is missing the required .runtimeconfig.json file." 1>&2
exit 106
fi
RUNTIMECONFIG_FILE="${RUNTIMECONFIG_FILES[0]}"
fi

DOTNET_ARGS+=("--depsfile" "${DEPS_FILE}"
"--runtimeconfig" "${RUNTIMECONFIG_FILE}"
"/var/runtime/Amazon.Lambda.RuntimeSupport.dll" "${LAMBDA_HANDLER}")
fi


# To support runtime wrapper scripts
# https://docs.aws.amazon.com/lambda/latest/dg/runtimes-modify.html#runtime-wrapper
if [ -z "${AWS_LAMBDA_EXEC_WRAPPER}" ]; then
if [ ${EXECUTABLE_BINARY_EXIST} = true ]; then
exec "${EXECUTABLE_BINARY}"
else
exec "${DOTNET_BIN}" "${DOTNET_EXEC}" "${DOTNET_ARGS[@]}"
fi
else
if [ ! -f "${AWS_LAMBDA_EXEC_WRAPPER}" ]; then
echo "${AWS_LAMBDA_EXEC_WRAPPER}: does not exist"
exit 127
fi
if [ ! -x "${AWS_LAMBDA_EXEC_WRAPPER}" ]; then
echo "${AWS_LAMBDA_EXEC_WRAPPER}: is not an executable"
exit 126
fi
if [ ${EXECUTABLE_BINARY_EXIST} = true ]; then
exec -- "${AWS_LAMBDA_EXEC_WRAPPER}" "${EXECUTABLE_BINARY}"
else
exec -- "${AWS_LAMBDA_EXEC_WRAPPER}" "${DOTNET_BIN}" "${DOTNET_EXEC}" "${DOTNET_ARGS[@]}"
fi
fi