The library can't handle async generators

[alexted]

Based on the example FastAPI + SQLAlchemy example from the documentation i made a myself implementation for work with connections of Async SQLA Core. But I came across the fact that your library does not support working with asynchronous generators. Could you please fix it somehow?
Here, here's my code:

from sqlalchemy.ext.asyncio import create_async_engine, AsyncEngine

from src.app.config import AppConfig


class Database:
    def __init__(self, config: AppConfig) -> None:
        self._engine: AsyncEngine = create_async_engine(config.DB_DSN)

    async def get_connection(self)  -> AsyncConnection:
        async with self._engine.begin() as connection:
            yield connection

class Container(containers.DeclarativeContainer):
    wiring_config = containers.WiringConfiguration(packages=["src.api"])

    config = providers.Configuration(pydantic_settings=[AppConfig(_env_file=".env")])
    app_cfg = config.get_pydantic_settings()[0]

    logging = providers.Resource(dictConfig, get_logging_config(app_cfg))

    db = providers.Resource(Database, config=app_cfg)

    language_repository = providers.Factory(
        repositories.LanguageRepository,
        connection=db.provided.get_connection.call(),
    )
    ...

I expect to get AsyncConnection, when initializing language_repository, but in fact I get <async_generator object Database.get_connection at 0x7ff63b913c40> for some reason. It seems that this should not be the case. I have to additionally do await anext(self._connection) in my repositories to get a live sqla connection. I also tried doing connection=db.provided.provided.get_connection.call().provided.__anext__.call(), but for some reason I get an inactive sqla connection which is impossible to work with.

[colonelpanic8]

I hit this as well, and I think that this is supposed to be a feature. You can avoid it by wrapping with a class or another object that is not a coroutine

[alexted]

I hit this as well, and I think that this is supposed to be a feature. You can avoid it by wrapping with a class or another object that is not a coroutine

Why do you think this it's not a bug? It's clearly a bug!

[colonelpanic8]

Why do you think this it's not a bug? It's clearly a bug!

it's not. If you read the code you will see that they clearly do this intentionally.

I think the reason is probably something like:

Well if you are exectuting the fetching of a dependency, its not at all clear on which event loop the asynchronous call is supposed to execute. Often, you would want it to execute on the current event loop, but that may not always be the case.

[alexted]

@rmk135 , could you pay attention to this issue? I really need to be able to do this:

async def get_db_session(
    async_session_factory: Annotated[AsyncEngine, Depends(get_session_factory)],
) -> AsyncGenerator[AsyncSession]:
    async with async_session_factory.begin() as session:
        yield session

[smandava]

u can avoid it by wrapping with a class or another object that is not a coroutine

@colonelpanic8 can you please provide a sample code that wraps with a class or another object that is not a coroutine?

[colonelpanic8]

it gets really hairy and nasty and I wouldn't necessarily recommend using this because you can definitely shoot yourself in the foot with it, but here you go:

async def _async_cm_enter(fn, *args, **kwargs):
    async def nested():
        async with fn(*args, **kwargs) as value:
            yield value

    coroutine = nested()

    value = await coroutine.__anext__()
    return coroutine, value


async def _async_cm_exit(coroutine):
    try:
        await coroutine.__anext__()
    except StopAsyncIteration:
        pass


@contextmanager
def async_cm_to_sync_cm(fn, loop, *args, **kwargs):
    coroutine, value = asyncio.run_coroutine_threadsafe(
        _async_cm_enter(fn, *args, **kwargs), loop
    ).result()

    try:
        yield value
    finally:
        try:
            asyncio.run_coroutine_threadsafe(_async_cm_exit(coroutine), loop).result(
                timeout=4
            )
        except TimeoutError as e:
            logger.info("Timed out waiting for _async_cm_exit", exc_info=e)


def SyncAsyncContextManagerResource(fn, *rargs, **rkwargs):
    def _wrapped(loop, *args, **kwargs):
        return async_cm_to_sync_cm(fn, loop, *args, **kwargs)

    return ContextManagerResource(_wrapped, *rargs, **rkwargs)

[ZipFile]

Hey! I just wanted to give you a quick heads-up. I'll be assisting @rmk135 in getting the project back on track. Our current priority is to work through the large backlog of issues and PRs by the end of the year to determine what we should focus on first.

As being mentioned earlier, given the current architecture, this issue is more of feature request than a bug. Async support is something that can improved, and something I've had to deal with in my projects, so it is likely to be prioritized.

[alexted]

Well, thanks for your work, we will wait for news!
I think this feature is really important, because nowadays it is impossible to imagine web programming in python not in the asynchronous paradigm. The widespread use of SQLAlchemy 2.x also reinforces the need for this feature.

[quirinoflavio]

@alexted How did you get through it?

[alexted]

@ZipFile could you give an estimate of when support for asynchronous generators might be added? Even a rough guess would help. Right now, this is the only thing preventing our team from using this library.
P.S. If you need a good code example, check out this microservice template - it has everything you need (just set alembic=yes and sqlalchemy=yes in the wizard).

[ZipFile]

Hey! I've been looking into this issue today. We indeed support async gens in resource providers, but there are few unhandled edge cases that results in behavior you're observing. I'm still trying to figure out proper fix, but in the mean time, try using Asynchronous subclass initializer for your resources. Hopefully ETA is next weekend (+ however long it would take to publish release), this issue was pain in the ass to isolate and debug.

Also, on unrelated note, seems like your template does not strictly follow src-layout, as you're not supposed to do from src.something import ... but rather from somethhing import .... Check official setuptools doc on package discovery, you can give it few hints when project layout auto-discovery fails.