-
Notifications
You must be signed in to change notification settings - Fork 113
Add S3EventNotifier
example
#477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
a565ae4
Add `S3EventNotifier` example
ptoffy b790508
Address review
ptoffy e67650a
Leftover
ptoffy 32b7459
Add back foundation
ptoffy 50e510b
Switch to standard Foundation
ptoffy 0ad4533
minor adjustments about Region usage
sebsto e3757aa
set region as envirnment variable
sebsto 1297c81
use correct markdown
sebsto File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
.DS_Store | ||
/.build | ||
/.index-build | ||
/Packages | ||
xcuserdata/ | ||
DerivedData/ | ||
.swiftpm/configuration/registries.json | ||
.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata | ||
.netrc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
// swift-tools-version: 6.0 | ||
import PackageDescription | ||
|
||
// needed for CI to test the local version of the library | ||
import struct Foundation.URL | ||
|
||
let package = Package( | ||
name: "CSVUploadAPINotificationLambda", | ||
platforms: [.macOS(.v15)], | ||
dependencies: [ | ||
.package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", branch: "main"), | ||
.package(url: "https://github.com/swift-server/swift-aws-lambda-events", branch: "main"), | ||
.package(url: "https://github.com/swift-server/async-http-client.git", from: "1.24.0"), | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
], | ||
targets: [ | ||
.executableTarget( | ||
name: "CSVUploadAPINotificationLambda", | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
dependencies: [ | ||
.product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"), | ||
.product(name: "AWSLambdaEvents", package: "swift-aws-lambda-events"), | ||
.product(name: "AsyncHTTPClient", package: "async-http-client"), | ||
] | ||
) | ||
] | ||
) | ||
|
||
if let localDepsPath = Context.environment["LAMBDA_USE_LOCAL_DEPS"], | ||
localDepsPath != "", | ||
let v = try? URL(fileURLWithPath: localDepsPath).resourceValues(forKeys: [.isDirectoryKey]), | ||
v.isDirectory == true | ||
{ | ||
// when we use the local runtime as deps, let's remove the dependency added above | ||
let indexToRemove = package.dependencies.firstIndex { dependency in | ||
if case .sourceControl( | ||
name: _, | ||
location: "https://github.com/swift-server/swift-aws-lambda-runtime.git", | ||
requirement: _ | ||
) = dependency.kind { | ||
return true | ||
} | ||
return false | ||
} | ||
if let indexToRemove { | ||
package.dependencies.remove(at: indexToRemove) | ||
} | ||
|
||
// then we add the dependency on LAMBDA_USE_LOCAL_DEPS' path (typically ../..) | ||
print("[INFO] Compiling against swift-aws-lambda-runtime located at \(localDepsPath)") | ||
package.dependencies += [ | ||
.package(name: "swift-aws-lambda-runtime", path: localDepsPath) | ||
] | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# S3 Event Notifier | ||
|
||
This example demonstrates how to create a Lambda that notifies an API of an S3 event in a bucket. | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Code | ||
|
||
In this example the lambda function receives an `S3Event` object from the `AWSLambdaEvents` library as input object instead of a `APIGatewayV2Request`. The `S3Event` object contains all the information about the S3 event that triggered the lambda, but what we are interested in is the bucket name and the object key, which are inside of a notification `Record`. The object contains an array of records, however since the lambda is triggered by a single event, we can safely assume that there is only one record in the array: the first one. Inside of this record, we can find the bucket name and the object key: | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```swift | ||
guard let s3NotificationRecord = event.records.first else { | ||
throw LambdaError.noNotificationRecord | ||
} | ||
|
||
let bucket = s3NotificationRecord.s3.bucket.name | ||
let key = s3NotificationRecord.s3.object.key.replacingOccurrences(of: "+", with: " ") | ||
``` | ||
|
||
The key is URL encoded, so we replace the `+` with a space. | ||
|
||
Once the event is decoded, the lambda sends a POST request to an API endpoint with the bucket name and the object key as parameters. The API URL is set as an environment variable. | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Build & Package | ||
|
||
To build & archive the package you can use the following commands: | ||
|
||
```bash | ||
swift build | ||
swift package archive --allow-network-connections docker | ||
``` | ||
|
||
If there are no errors, a ZIP file should be ready to deploy, located at `.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/S3EventNotifier/S3EventNotifier.zip`. | ||
|
||
## Deploy | ||
|
||
To deploy the Lambda function, you can use the `aws` command line: | ||
|
||
```bash | ||
aws lambda create-function \ | ||
--function-name S3EventNotifier \ | ||
--zip-file fileb://.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/S3EventNotifier/S3EventNotifier.zip \ | ||
--runtime provided.al2 \ | ||
--handler provided \ | ||
--architectures arm64 \ | ||
--role arn:aws:iam::<YOUR_ACCOUNT_ID>:role/lambda_basic_execution | ||
``` | ||
|
||
The `--architectures` flag is only required when you build the binary on an Apple Silicon machine (Apple M1 or more recent). It defaults to `x64`. | ||
|
||
Be sure to replace <YOUR_ACCOUNT_ID> with your actual AWS account ID (for example: 012345678901). | ||
|
||
> [!WARNING] | ||
> You will have to set up an S3 bucket and configure it to send events to the lambda function. This is not covered in this example. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add minimum command line instructions to do so
Something like this (not tested)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
//===----------------------------------------------------------------------===// | ||
// | ||
// This source file is part of the SwiftAWSLambdaRuntime open source project | ||
// | ||
// Copyright (c) 2024 Apple Inc. and the SwiftAWSLambdaRuntime project authors | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// Licensed under Apache License v2.0 | ||
// | ||
// See LICENSE.txt for license information | ||
// See CONTRIBUTORS.txt for the list of SwiftAWSLambdaRuntime project authors | ||
// | ||
// SPDX-License-Identifier: Apache-2.0 | ||
// | ||
//===----------------------------------------------------------------------===// | ||
|
||
import AWSLambdaEvents | ||
import AWSLambdaRuntime | ||
import AsyncHTTPClient | ||
|
||
#if canImport(FoundationEssentials) | ||
import FoundationEssentials | ||
#else | ||
import Foundation | ||
#endif | ||
|
||
let httpClient = HTTPClient.shared | ||
|
||
enum LambdaError: Error { | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
case noNotificationRecord | ||
case missingEnvVar(name: String) | ||
|
||
var description: String { | ||
switch self { | ||
case .noNotificationRecord: | ||
"No notification record in S3 event" | ||
case .missingEnvVar(let name): | ||
"Missing env var named \(name)" | ||
} | ||
} | ||
} | ||
|
||
let runtime = LambdaRuntime { (event: S3Event, context: LambdaContext) async throws -> APIGatewayV2Response in | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
do { | ||
context.logger.debug("Received S3 event: \(event)") | ||
|
||
guard let s3NotificationRecord = event.records.first else { | ||
throw LambdaError.noNotificationRecord | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
let bucket = s3NotificationRecord.s3.bucket.name | ||
let key = s3NotificationRecord.s3.object.key.replacingOccurrences(of: "+", with: " ") | ||
|
||
guard let apiURL = ProcessInfo.processInfo.environment["API_URL"] else { | ||
throw LambdaError.missingEnvVar(name: "API_URL") | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
let body = """ | ||
{ | ||
"bucket": "\(bucket)", | ||
"key": "\(key)" | ||
} | ||
""" | ||
|
||
context.logger.debug("Sending request to \(apiURL) with body \(body)") | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
var request = HTTPClientRequest(url: "\(apiURL)/upload-complete/") | ||
request.method = .POST | ||
request.headers = [ | ||
"Content-Type": "application/json" | ||
] | ||
request.body = .bytes(.init(string: body)) | ||
|
||
let response = try await httpClient.execute(request, timeout: .seconds(30)) | ||
return APIGatewayV2Response( | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
statusCode: .ok, | ||
body: "Lambda terminated successfully. API responded with: Status: \(response.status), Body: \(response.body)" | ||
) | ||
} catch let error as LambdaError { | ||
context.logger.error("\(error.description)") | ||
return APIGatewayV2Response(statusCode: .internalServerError, body: "[ERROR] \(error.description)") | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} catch { | ||
context.logger.error("\(error)") | ||
return APIGatewayV2Response(statusCode: .internalServerError, body: "[ERROR] \(error)") | ||
ptoffy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
} | ||
|
||
try await runtime.run() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.