Significant performance hit when using async resolvers

[kevinvalk]

During development (Starlette + Ariadne) of our product I noticed significant performance degradation when GraphQL responses got significant long (1000+ entities in a list). I started profiling and drilling into the issue and I pinpointed it to async resolvers. Whenever a resolver is async and it is called a lot (100.000) you can see significant slowdowns 4x-7x, even if there is nothing async to it.

The question that I ended up with, is this a limitation of Python asyncio or how the results are gathered for async fields in graphql execute?

Any insight/help is greatly appreciated as we really need more performance and changing to sync is not really an option (nor is rewriting it to another language) 😭

import asyncio
from dataclasses import dataclass

from graphql import (
    GraphQLField,
    GraphQLList,
    GraphQLObjectType,
    GraphQLSchema,
    GraphQLString,
    graphql,
)

# On a MacBook Pro M1 Max this takes around 44 seconds in "async" case
# and "only" 7 seconds when async field is removed from the query
TOTAL_PEOPLE = 1000000


@dataclass
class Person:
    firstName: str
    lastName: str


def resolve_people(*_):
    return [Person(f"Jane{i}", "Do") for i in range(0, TOTAL_PEOPLE)]


def resolve_fullname(person: Person, *_):
    return f"{person.firstName} {person.lastName}"


async def async_resolve_fullname(person: Person, *_):
    return f"{person.firstName} {person.lastName}"


PersonType = GraphQLObjectType(
    "Person",
    {
        "firstName": GraphQLField(GraphQLString),
        "lastName": GraphQLField(GraphQLString),
        "fullName": GraphQLField(GraphQLString, resolve=resolve_fullname),
        "asyncFullName": GraphQLField(GraphQLString, resolve=async_resolve_fullname),
    },
)

schema = GraphQLSchema(
    query=GraphQLObjectType(
        name="RootQueryType",
        fields={
            "people": GraphQLField(GraphQLList(PersonType), resolve=resolve_people)
        },
    )
)


async def main():
    result = await graphql(
        schema,
        """#graphql
        query {
            people {
                firstName
                lastName
                fullName
                asyncFullName # THIS IS THE SLOW PART
            }
        }
    """,
    )
    assert len(result.data["people"]) == TOTAL_PEOPLE


def run(callable, is_profile: bool = False):
    if not is_profile:
        return asyncio.run(callable())

    import yappi

    yappi.set_clock_type("WALL")
    with yappi.run():
        asyncio.run(callable())

    func_stats = yappi.get_func_stats()

    func_stats.save("callgrind.func_stats", "callgrind")
    with open("func_stats.txt", "w") as file:
        func_stats.print_all(
            file,
            {
                # We increase the column widths significantly
                0: ("name", 120),
                1: ("ncall", 10),
                2: ("tsub", 12),
                3: ("ttot", 12),
                4: ("tavg", 12),
            },
        )


if __name__ == "__main__":
    run(main, is_profile=False)

Versions:

• macOS: 13.2
• python 3.11
• graphql-core 3.2.3
• yappi 1.4.0

[Cito]

Thanks, will definitely look into that. For clarification: When you say degradation, then in comparison to what? Smaller results, sync resolvers, or earlier versions of GraphQL-core or Python?

[kevinvalk]

Compared with sync resolver.

The field fullName is connected to def resolve_fullname and asyncFullName to async def async_resolve_fullname. There is 4-7x performance difference (sync is faster) between those two resolvers with the only difference being async method.

[Cito]

Ok, thanks for clarification. I think #142 and #189 are related issues.

[JCatrielLopez]

Just to clarify, on #189 we didn't use async resolvers

[kevinvalk]

Did some performance tests and it seems asyncio.gather is HORRIBLY slow for cpu bound futures when you have many.
https://perfpy.com/240

So based on these results I created the following monkey patch that gives us in our current production code 2x speed improvement:

execute = import_module(".execution.execute", "graphql")
async def serial_gather(*futures):
    return [await future for future in futures]
execute.gather = serial_gather

NOTE: These tests are based on a CPU bound "resolver" so any optimizations have to be checked to a more normal workload with maybe a mix of CPU bound and IO bound resolvers.

That said, as @JCatrielLopez indicated, there is also slowdowns in sync code so I think there is a lot of performance left on the table. @Cito how could we tackle investigations into increasing performance of graphql-python library?

[jkimbo]

@kevinvalk thats a really good find and it would definitely help the cases with simple async resolvers. However it's worth pointing out that as soon as your resolvers are actually doing anything that might take some time (i.e. HTTP requests or file access) using the serial gather method will slow things down because the functions aren't running in parallel. I've written benchmark to show that (the async function sleeps for 50 milliseconds on every 1000th iteration): https://perfpy.com/242

There might be an argument to allowing developers to configure how async results are gathered but why are you using async functions for purely synchronous work anyway @kevinvalk ? Async functions will always have an overhead compared to sync functions.

[Cito]

Thank you all for your input. @kevinvalk The way that gather works in Python is that it creates an async task for each of the passed coroutines. If the overhead of running asyncio.create_task is high compared to the runtime of the coroutine itself, in this case creating a single random value with random.randint, then of course you will get a very poor performance. Normally you trade the overhead of creating a task against the advantage of running code concurrently, but in this case, since the code is pure CPU code that is blocking the event loop, you do not even gain anything from it as @jkimbo pointed out. With other words, the benchmarks just shows how much slower asyncio.create_task is compared to random.randint.

However, I think we all agree that there is room for improvement for supporting different strategies, maybe in form of optimization hints for the execution algorithm. At least, it should be possible to specify whether a serial or parallel execution strategy is used. Currently, only mutations are resolved serially, and you have no way to force queries to be executed serially (just like in GraphQl.js).

Note that GraphQL-core already detects when all (sub)field resolvers are synchronous and does not try to gather anything in this case. As @jkimbo wrote, using sync resolvers is therefore the simplest solution in such cases.

[jkimbo]

@Cito One optimisation that we could make is to never use asyncio.gather when there is only 1 coroutine since it wouldn't provide any benefit. It would help the example in this issue a lot but I'm not sure how helpful it would be for most real use cases. What do you think?

[Cito]

@jkimbo Yes, that's a simple thing I can implement. The additional check is probably cheap compared to the overhead of running gather.