Skip to content

feat(functions): PostgreSQL Python example #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ Table of Contents:
| **[Python Dependencies](functions/python-dependencies/README.md)** <br/> Example showing how to use Python requirements with Serverless Framework. | python310 | [Serverless Framework] |
| **[Redis TLS](functions/redis-tls/README.md)** <br/> How to connect a function to a Scaleway Redis cluster with TLS enabled. | python310 | [Serverless Framework] |
| **[Rust MNIST](functions/rust-mnist/README.md)** <br/> A Rust function to recognize hand-written digits with a simple neural network. | rust165 | [Serverless Framework] |
| **[PostgreSQL Python](functions/postgre-sql-python/README.md)** <br/> A Python function to perform a query on a PostgreSQL managed database. | python310 | [Serverless Framework] |
| **[Terraform Python](functions/terraform-python-example/README.md)** <br/> A Python function deployed with Terraform. | python310 | [Terraform] |

### 📦 Containers
Expand Down
61 changes: 61 additions & 0 deletions functions/postgre-sql-python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Interact with a PostgreSQL database using pythonic functions

## Requirements

This example assumes that you are familiar with some products of Scaleway's ecosystem:
* how serverless functions work. If needed, you can check [Scaleway official documentation](https://www.scaleway.com/en/docs/serverless/functions/quickstart/).
* how a managed database for PostgreSQL works, and especially how to create a database and create users with appropriate permissions. Please refer to scaleway's documentation [here](https://www.scaleway.com/en/docs/managed-databases/postgresql-and-mysql/quickstart/).

This example uses the Scaleway Serverless Framework Plugin. Please set up your environment with the requirements stated in the [Scaleway Serverless Framework Plugin](https://github.com/scaleway/serverless-scaleway-functions) before trying out the example.


## Context

This example shows how to connect to a managed PostgreSQL database and perform a query on it. This example can be extended to adding, deleting, or processing data within a database.


## Description

The function connects to a PostgreSQL database and performs an example query on it. This example uses Python 3.10 runtime. Used packages are specified in `package.json`. As a requirement for this example we use a pre-compiled [`psycopg2` version](https://www.psycopg.org/docs/install.html#install-from-source).

## Setup

### Create a managed PostgreSQL database

Create a PostgreSQL database and a user profile with appropriate access permissions.

### Fill environment variables

Fill your secrets within `serverless.yml` file:

```
secret:
PG_HOST: "your host IP address"
PG_USER: "your database username"
PG_DATABASE: "your database name"
PG_PASSWORD: "your database user password"
PG_PORT: "your database port"
PG_SSL_ROOT_CERT: "path to your database ssl certificate"
```

### Install npm modules

Once your environment is set up, you can install `npm` dependencies from `package.json` file using:

```
npm install
```

### Deploy and call the function

Then deploy your function and get its URL using:

```
./bin/deploy.sh
```

From the given function URL, you can run your function using:

```
curl <function URL>
```
7 changes: 7 additions & 0 deletions functions/postgre-sql-python/bin/deploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash

set -e

./bin/deps.sh

serverless deploy
11 changes: 11 additions & 0 deletions functions/postgre-sql-python/bin/deps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

set -e

# This command will run pip install with the given requirements.txt file inside
# a docker container compatible with our function runtimes, and pull the installed
# dependencies locally to your package directory. As these dependencies have been
# installed on top of alpine Linux with our compatible system libraries, you will
# be able to upload your source code and deploy your function properly.
PYTHON_VERSION=3.10
docker run --rm -v $(pwd):/home/app/function --workdir /home/app/function rg.fr-par.scw.cloud/scwfunctionsruntimes-public/python-dep:$PYTHON_VERSION pip install -r ./requirements.txt --target ./package
56 changes: 56 additions & 0 deletions functions/postgre-sql-python/handlers/handler.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import psycopg2
from psycopg2 import Error
import os
import logging

PG_HOST=os.environ.get('PG_HOST')
PG_USER=os.environ.get('PG_USER')
PG_DATABASE=os.environ.get('PG_DATABASE')
PG_PASSWORD=os.environ.get('PG_PASSWORD')
PG_PORT=os.environ.get('PG_PORT')
PG_SSL_ROOT_CERT=os.environ.get('PG_SSL_ROOT_CERT')
Comment on lines +6 to +11
Copy link
Contributor

@cyclimse cyclimse Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PG_HOST=os.environ.get('PG_HOST')
PG_USER=os.environ.get('PG_USER')
PG_DATABASE=os.environ.get('PG_DATABASE')
PG_PASSWORD=os.environ.get('PG_PASSWORD')
PG_PORT=os.environ.get('PG_PORT')
PG_SSL_ROOT_CERT=os.environ.get('PG_SSL_ROOT_CERT')
PG_HOST=os.environ["PG_HOST"]
PG_USER=os.environ["PG_USER"]
PG_DATABASE=os.environ["PG_DATABASE"]
PG_PASSWORD=os.environ["PG_PASSWORD"]
PG_PORT=os.environ["'PG_PORT"]
PG_SSL_ROOT_CERT=os.environ["PG_SSL_ROOT_CERT"]

I prefer os.getenv instead of os.environ.get. But because here those env vars are required, it's even better to catch the error early.


def handle(event, context):

try:
connection = psycopg2.connect(
database=PG_DATABASE,
user=PG_USER,
host=PG_HOST,
password=PG_PASSWORD,
port=PG_PORT,
sslmode="require",
sslrootcert=PG_SSL_ROOT_CERT,
)

except (Exception, Error) as error:
logging.error("Error while connecting to PostgreSQL database", error)
return {
"statusCode": 500,
"body": {
"message": "Error while connecting to PostgreSQL database, check function logs for more information"
}
}

try:
cursor = connection.cursor()
logging.info("Connected to Database")
cursor.execute("SELECT * FROM table LIMIT 10")
record = cursor.fetchone()
logging.info("Successfully fetched data")
cursor.close()
return {
"statusCode": 200,
"body": {
"message": record
}
}

except (Exception, Error) as error:
logging.error("Error while interacting with PostgreSQL", error)
return {
"statusCode": 500,
"body": {
"message": "Error while getting information from PostgreSQL database, check function logs for more information"
}
}
5 changes: 5 additions & 0 deletions functions/postgre-sql-python/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"devDependencies": {
"serverless-scaleway-functions": "^0.4.6"
}
}
1 change: 1 addition & 0 deletions functions/postgre-sql-python/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
psycopg2-binary == 2.9
27 changes: 27 additions & 0 deletions functions/postgre-sql-python/serverless.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
service: get-all-from-database

configValidationMode: off

provider:
name: scaleway
runtime: python310
secret:
PG_HOST: "your database host IP address"
PG_USER: "your database username"
PG_DATABASE: "your database name"
PG_PASSWORD: "your database user password"
PG_PORT: "your database port"
PG_SSL_ROOT_CERT: "path to your database ssl certificate"

plugins:
- serverless-scaleway-functions

package:
patterns:
- '!node_modules/**'
- '!.gitignore'
- '!.git/**'

functions:
first:
handler: handlers/handler.handle