Skip to content

Python codegen support #923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/ci-python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: python
on: [push, pull_request]
jobs:

build:
name: test
runs-on: ubuntu-latest

services:
postgres:
image: postgres:11
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install python dependencies
working-directory: ./examples/python
run: |
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
- name: Test python code
working-directory: ./examples/python
env:
PG_USER: postgres
PG_HOST: localhost
PG_DATABASE: postgres
PG_PASSWORD: postgres
PG_PORT: ${{ job.services.postgres.ports['5432'] }}
run: |
pytest src/tests
6 changes: 6 additions & 0 deletions examples/python/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pytest~=6.2.2
pytest-asyncio~=0.14.0
psycopg2-binary~=2.8.6
asyncpg~=0.21.0
pydantic~=1.7.3
sqlc-python-runtime~=1.0.0
49 changes: 49 additions & 0 deletions examples/python/sqlc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"version": "2",
"sql": [
{
"schema": "../authors/postgresql/schema.sql",
"queries": "../authors/postgresql/query.sql",
"engine": "postgresql",
"gen": {
"python": {
"out": "src/authors",
"package": "authors"
}
}
},
{
"schema": "../booktest/postgresql/schema.sql",
"queries": "../booktest/postgresql/query.sql",
"engine": "postgresql",
"gen": {
"python": {
"out": "src/booktest",
"package": "booktest"
}
}
},
{
"schema": "../jets/schema.sql",
"queries": "../jets/query-building.sql",
"engine": "postgresql",
"gen": {
"python": {
"out": "src/jets",
"package": "jets"
}
}
},
{
"schema": "../ondeck/postgresql/schema",
"queries": "../ondeck/postgresql/query",
"engine": "postgresql",
"gen": {
"python": {
"out": "src/ondeck",
"package": "ondeck"
}
}
}
]
}
Empty file.
15 changes: 15 additions & 0 deletions examples/python/src/authors/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Code generated by sqlc. DO NOT EDIT.
from typing import Optional

import pydantic


# Enums

# Models
Comment on lines +7 to +9
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't generate this comments, they're mainly just noise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, these are leftover from my initial single mega-file implementation, I'll remove them.

class Author(pydantic.BaseModel):
id: int
name: str
bio: Optional[str]


92 changes: 92 additions & 0 deletions examples/python/src/authors/query.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Code generated by sqlc. DO NOT EDIT.
from typing import AsyncIterator, Awaitable, Iterator, Optional, overload

import sqlc_runtime as sqlc

from authors import models


CREATE_AUTHOR = """-- name: create_author :one
INSERT INTO authors (
name, bio
) VALUES (
$1, $2
)
RETURNING id, name, bio
"""


DELETE_AUTHOR = """-- name: delete_author :exec
DELETE FROM authors
WHERE id = $1
"""


GET_AUTHOR = """-- name: get_author :one
SELECT id, name, bio FROM authors
WHERE id = $1 LIMIT 1
"""


LIST_AUTHORS = """-- name: list_authors :many
SELECT id, name, bio FROM authors
ORDER BY name
"""
Comment on lines +31 to +34
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like to generate the query strings closed to the functions themselves so it's easier to read. If I'm looking at list_authors, I can then see the query it's referencing without having to jump around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's nice to keep them close together, but how would this work if the query functions are in a class? I could define these constants as class properties above each method, but I can't say I've seen other Python code that does that, class properties usually go at the top of the class definition.



@overload
def create_author(conn: sqlc.Connection, name: str, bio: Optional[str]) -> Optional[models.Author]:
pass


@overload
def create_author(conn: sqlc.AsyncConnection, name: str, bio: Optional[str]) -> Awaitable[Optional[models.Author]]:
pass


def create_author(conn: sqlc.GenericConnection, name: str, bio: Optional[str]) -> sqlc.ReturnType[Optional[models.Author]]:
return conn.execute_one_model(models.Author, CREATE_AUTHOR, name, bio)
Comment on lines +37 to +48
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never used or seen the @overload operator. How common is it? Are there other ways to expose a similar sync / async function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how common it is, I'm using it here so a single function can be used with both sync and async engines and type checkers properly understand it. It's definitely something I cooked up myself though, I haven't seen this specific pattern for sync+async support anywhere else. Without it I'd have to define separate sync and async functions, and with bare functions I definitely didn't want to just prefix (or suffix) every function with async_.

However, if the query functions are moved into classes it's a lot simpler to just define two classes, one with sync functions and another with async functions, no method prefix/suffix is needed since they are contained within separate classes. The generation of async (or sync) query classes can also be placed behind a config option, so users who don't care about async (or sync) support don't get that generated code.



@overload
def delete_author(conn: sqlc.Connection, id: int) -> None:
pass


@overload
def delete_author(conn: sqlc.AsyncConnection, id: int) -> Awaitable[None]:
pass


def delete_author(conn: sqlc.GenericConnection, id: int) -> sqlc.ReturnType[None]:
return conn.execute_none(DELETE_AUTHOR, id)


@overload
def get_author(conn: sqlc.Connection, id: int) -> Optional[models.Author]:
pass


@overload
def get_author(conn: sqlc.AsyncConnection, id: int) -> Awaitable[Optional[models.Author]]:
pass


def get_author(conn: sqlc.GenericConnection, id: int) -> sqlc.ReturnType[Optional[models.Author]]:
return conn.execute_one_model(models.Author, GET_AUTHOR, id)


@overload
def list_authors(conn: sqlc.Connection) -> Iterator[models.Author]:
pass


@overload
def list_authors(conn: sqlc.AsyncConnection) -> AsyncIterator[models.Author]:
pass


def list_authors(conn: sqlc.GenericConnection) -> sqlc.IteratorReturn[models.Author]:
return conn.execute_many_model(models.Author, LIST_AUTHORS)


Empty file.
31 changes: 31 additions & 0 deletions examples/python/src/booktest/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Code generated by sqlc. DO NOT EDIT.
from typing import List
import datetime
import enum

import pydantic


# Enums
class BookType(str, enum.Enum):
FICTION = "FICTION"
NONFICTION = "NONFICTION"


# Models
class Author(pydantic.BaseModel):
author_id: int
name: str


class Book(pydantic.BaseModel):
book_id: int
author_id: int
isbn: str
book_type: BookType
title: str
year: int
available: datetime.datetime
tags: List[str]


Loading