Skip to content

Commit d9d4a5d

Browse files
authored
feat: add node s3 upload example (#27)
* feat: add node s3 upload example * chore: run prettierjs * chore: bad package.json name * chore: missing endline in package.json
1 parent 63788e9 commit d9d4a5d

File tree

8 files changed

+129
-5
lines changed

8 files changed

+129
-5
lines changed

.gitignore

Lines changed: 0 additions & 1 deletion
This file was deleted.

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,12 +34,12 @@ Table of Contents:
3434
| **[CORS Python](functions/cors-python/README.md)** <br/> A Python function which allows CORS requests. | python310 | [Serverless Framework] |
3535
| **[CORS Rust](functions/cors-rust/README.md)** <br/> A Rust function which allows CORS requests. | rust165 | [Serverless Framework] |
3636
| **[Go SQS Publish](functions/go-mnq-sqs-publish/README.md)** <br/> A Go function to publish messages to SQS. | go118 | [Serverless Framework] |
37-
| **[Go MultiPart Upload to S3](functions/go-mnq-sqs-publish/README.md)** <br/> A function to upload file from form-data to S3. | go119 | [Serverless Framework] |
38-
| **[CORS Rust](functions/cors-rust/README.md)** <br/> A Rust function which allows CORS requests. | rust165 | [Serverless Framework] |
37+
| **[Go MultiPart Upload to S3](functions/go-upload-file-s3-multipart)** <br/> A function to upload file from form-data to S3. | go119 | [Serverless Framework] |
3938
| **[Image Transform](functions/image-transform-node/README.md)** <br/> A function that resizes images from an S3 bucket. | node16 | [Serverless Framework] |
39+
| **[Node MultiPart Upload to S3](functions/node-upload-file-s3-multipart/README.md)** <br/> A function to upload file from form-data to S3. | node19 | [Serverless Framework] |
4040
| **[Python ChatBot](functions/python-dependencies/README.md)** <br/> A chatbot example with ChatterBot. | python310 | [Serverless Framework] |
4141
| **[Python Dependencies](functions/python-dependencies/README.md)** <br/> Example showing how to use Python requirements with Serverless Framework. | python310 | [Serverless Framework] |
42-
| **[Redis TLS](functions/redis-tls/README.md)** <br/> How to connect a function to a Scaleway Redis cluster with TLS enabled. | python310 | [Serverless Framework] |
42+
| **[Redis TLS](functions/redis-tls/README.md)** <br/> How to connect a function to a Scaleway Redis cluster with TLS enabled. | python310 | [Terraform] |
4343
| **[Rust MNIST](functions/rust-mnist/README.md)** <br/> A Rust function to recognize hand-written digits with a simple neural network. | rust165 | [Serverless Framework] |
4444
| **[Terraform Python](functions/terraform-python-example/README.md)** <br/> A Python function deployed with Terraform. | python310 | [Terraform] |
4545
| **[PostgeSQL Node](functions/postgre-sql-node/README.md)** <br/> A Node function to connect and interact with PostgreSQL database. | node18 | [Serverless Framework] |

functions/go-upload-file-s3-multipart/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ S3_BUCKET_NAME=
2020
S3_REGION= # ex: fr-par
2121
```
2222

23-
If s3 not enabled the file will be saved on the ephemeral storage of your function.
23+
If s3 is not enabled the file will be saved on the ephemeral storage of your function.
2424

2525
```sh
2626
go get
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules/
2+
package-lock.json
3+
.serverless/
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Node function to read files and upload to S3
2+
3+
This function does the following steps :
4+
5+
* Read a file from an HTTP request
6+
* Send the file to an S3 bucket
7+
8+
## Requirements
9+
10+
If you want to enable S3 upload, ensure to create a bucket and have the following secrets variables available in your environment:
11+
12+
```env
13+
S3_REGION = # Default: fr-par
14+
SCW_ACCESS_KEY =
15+
SCW_SECRET_KEY =
16+
BUCKET_NAME =
17+
```
18+
19+
To call the function (replace `README.md` with the file you want to upload):
20+
21+
```sh
22+
curl -X POST -F "data=@README.md" -F "another=@package.json" http://localhost:8080
23+
```
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
"use strict";
2+
3+
import { STATUS_CODES } from "http";
4+
5+
import { S3 } from "@aws-sdk/client-s3";
6+
import { parse } from "parse-multipart-data";
7+
8+
const BUCKET_NAME = process.env.BUCKET_NAME;
9+
const S3_REGION = process.env.S3_REGION;
10+
const ACCESS_KEY_ID = process.env.ACCESS_KEY_ID;
11+
const SECRET_KEY = process.env.SECRET_KEY;
12+
13+
// Create S3 service object
14+
const s3 = new S3({
15+
endpoint: `https://s3.${S3_REGION}.scw.cloud`,
16+
region: S3_REGION,
17+
credentials: {
18+
accessKeyId: ACCESS_KEY_ID,
19+
secretAccessKey: SECRET_KEY,
20+
},
21+
});
22+
23+
const uploadFormDataS3 = async (event, _context, cb) => {
24+
if (
25+
!event.headers["Content-Type"] ||
26+
!event.headers["Content-Type"].startsWith("multipart/form-data")
27+
) {
28+
return { statusCode: 400, body: STATUS_CODES[400] };
29+
}
30+
31+
// Get the boundary from the Content-Type header
32+
const boundary = event.headers["Content-Type"].split("=").pop();
33+
const parts = parse(Buffer.from(event["body"], "utf-8"), boundary);
34+
35+
let promises = [];
36+
for (const part of parts) {
37+
console.log(`Saving object ${part.filename} to bucket ${BUCKET_NAME}...`);
38+
promises.push(
39+
s3.putObject({ Bucket: BUCKET_NAME, Key: part.filename, Body: part.data })
40+
);
41+
}
42+
43+
try {
44+
await Promise.all(promises);
45+
return {
46+
statusCode: 200,
47+
body: `Successfully uploaded ${parts
48+
.map((f) => f.filename)
49+
.join(",")} to ${BUCKET_NAME}`,
50+
};
51+
} catch (err) {
52+
return {
53+
statusCode: 500,
54+
body: err.message,
55+
};
56+
}
57+
};
58+
59+
export { uploadFormDataS3 };
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"name": "s3-form-data",
3+
"version": "1.0.0",
4+
"main": "handler.js",
5+
"type": "module",
6+
"keywords": [],
7+
"author": "",
8+
"license": "ISC",
9+
"dependencies": {
10+
"@aws-sdk/client-s3": "^3.289.0",
11+
"parse-multipart-data": "^1.5.0"
12+
},
13+
"devDependencies": {
14+
"serverless-scaleway-functions": "^0.4.4"
15+
},
16+
"description": ""
17+
}
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
service: s3-form-data
2+
configValidationMode: off
3+
provider:
4+
name: scaleway
5+
runtime: node19
6+
env:
7+
BUCKET_NAME: ${env:BUCKET_NAME}
8+
S3_REGION: ${env:S3_REGION, "fr-par"}
9+
secret:
10+
ACCESS_KEY_ID: ${env:SCW_ACCESS_KEY}
11+
SECRET_KEY: ${env:SCW_SECRET_KEY}
12+
13+
plugins:
14+
- serverless-scaleway-functions
15+
16+
package:
17+
patterns:
18+
- "!.gitignore"
19+
- "!.git/**"
20+
21+
functions:
22+
upload:
23+
handler: handler.uploadFormDataS3

0 commit comments

Comments
 (0)