Skip to content

Commit a17ca7c

Browse files
ArchmongerArchmonger
committed
use_query docs
1 parent 17c5bd8 commit a17ca7c

File tree

6 files changed

+189
-125
lines changed

6 files changed

+189
-125
lines changed

‎docs/includes/orm.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
<!--orm-excp-start-->
2+
3+
Due to Django's ORM design, database queries must be deferred using hooks. Otherwise, you will see a `SynchronousOnlyOperation` exception.
4+
5+
These `SynchronousOnlyOperation` exceptions may be resolved in a future version of Django containing an asynchronous ORM. However, it is best practice to always perform ORM calls in the background via hooks.
6+
7+
<!--orm-excp-end-->
8+
9+
<!--orm-fetch-start-->
10+
11+
By default, automatic recursive fetching of `ManyToMany` or `ForeignKey` fields is enabled within the default `QueryOptions.postprocessor`. This is needed to prevent `SynchronousOnlyOperation` exceptions when accessing these fields within your IDOM components.
12+
13+
<!--orm-fetch-end-->

‎docs/src/dictionary.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ changelog
66
async
77
pre
88
prefetch
9+
prefetching
910
preloader
1011
whitespace
1112
refetch

‎docs/src/features/hooks.md

Lines changed: 117 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -64,87 +64,120 @@ The function you provide into this hook must return either a `Model` or `QuerySe
6464
| --- | --- |
6565
| `Query[_Result | None]` | An object containing `loading`/`error` states, your `data` (if the query has successfully executed), and a `refetch` callable that can be used to re-run the query. |
6666

67-
??? question "Can I make ORM calls without hooks?"
67+
??? question "How can I provide arguments to my query function?"
68+
69+
`*args` and `**kwargs` can be provided to your query function via `use_query` parameters.
70+
71+
=== "components.py"
6872

69-
Due to Django's ORM design, database queries must be deferred using hooks. Otherwise, you will see a `SynchronousOnlyOperation` exception.
73+
```python
74+
from idom import component
75+
from django_idom.hooks import use_query
7076

71-
This compatibility may be resolved in a future version of Django containing an asynchronous ORM. However, it is still best practice to always perform ORM calls in the background via `use_query`.
77+
def example_query(value:int, other_value:bool = False):
78+
...
79+
80+
@component
81+
def my_component():
82+
query = use_query(
83+
example_query,
84+
123,
85+
other_value=True,
86+
)
87+
88+
...
89+
```
90+
91+
??? question "Why does the example `get_items` function return `TodoItem.objects.all()`?"
92+
93+
This was a technical design decision to based on [Apollo's `useQuery` hook](https://www.apollographql.com/docs/react/data/queries/), but ultimately helps avoid Django's `SynchronousOnlyOperation` exceptions.
94+
95+
The `use_query` hook ensures the provided `Model` or `QuerySet` executes all [deferred](https://docs.djangoproject.com/en/dev/ref/models/instances/#django.db.models.Model.get_deferred_fields)/[lazy queries](https://docs.djangoproject.com/en/dev/topics/db/queries/#querysets-are-lazy) safely prior to reaching your components.
7296

7397
??? question "Can this hook be used for things other than the Django ORM?"
7498

75-
If you...
99+
{% include-markdown "../../includes/orm.md" start="<!--orm-fetch-start-->" end="<!--orm-fetch-end-->" %}
100+
101+
However, if you...
76102

77103
1. Want to use this hook to defer IO intensive tasks to be computed in the background
78104
2. Want to to utilize `use_query` with a different ORM
79105

80-
... then you can disable all postprocessing behavior by modifying the `postprocessor` parameter in `QueryOptions`.
106+
... then you can disable all postprocessing behavior by modifying the `QueryOptions.postprocessor` parameter. In the example below, we will set the `postprocessor` to a function that takes one argument and returns nothing.
81107

82-
```python
83-
from django_idom.types import QueryOptions
84-
from django_idom.hooks import use_query
108+
=== "components.py"
85109

86-
def io_intensive_operation():
87-
"""This is an example function call that does something IO intensive, but can
88-
potentially fail to execute."""
89-
...
110+
```python
111+
from idom import component
112+
from django_idom.types import QueryOptions
113+
from django_idom.hooks import use_query
90114

91-
@component
92-
def todo_list():
93-
query = use_query(
94-
io_intensive_operation,
95-
QueryOptions(
96-
# By setting the postprocessor to a function that takes one argument
97-
# and returns None, we can disable postprocessing behavior.
98-
postprocessor=lambda data: None,
99-
),
100-
)
115+
def execute_io_intensive_operation():
116+
"""This is an example query function that does something IO intensive."""
117+
pass
101118

102-
if query.loading or query.error:
103-
return None
119+
@component
120+
def todo_list():
121+
query = use_query(
122+
execute_io_intensive_operation,
123+
QueryOptions(
124+
# By setting the postprocessor to a function that takes one argument
125+
# and returns None, we can disable postprocessing behavior.
126+
postprocessor=lambda data: None,
127+
),
128+
)
104129

105-
return str(query.data)
106-
```
130+
if query.loading or query.error:
131+
return None
107132

108-
??? question "Can this hook automatically fetch `ManyToMany` fields or `ForeignKey` relationships?"
133+
return str(query.data)
134+
```
109135

110-
By default, automatic recursive fetching of `ManyToMany` or `ForeignKey` fields is disabled for performance reasons.
136+
??? question "How can I prevent this hook from recursively fetching `ManyToMany` fields or `ForeignKey` relationships?"
111137

112-
If you wish you enable this feature, you can modify the `postprocessor_kwargs` parameter in `QueryOptions`.
138+
{% include-markdown "../../includes/orm.md" start="<!--orm-fetch-start-->" end="<!--orm-fetch-end-->" %}
113139

114-
```python
115-
from example_project.my_app.models import MyModel
116-
from django_idom.types import QueryOptions
117-
from django_idom.hooks import use_query
140+
However, if you have deep nested trees of relational data, this may not be a desirable behavior. You may prefer to manually fetch these relational fields using a second `use_query` hook.
118141

119-
def model_with_relationships():
120-
"""This is an example function that gets `MyModel` that has a ManyToMany field, and
121-
additionally other models that have formed a ForeignKey association to `MyModel`.
142+
You can disable the prefetching behavior of the default `postprocessor` (located at `django_idom.utils.django_query_postprocessor`) via the `QueryOptions.postprocessor_kwargs` parameter.
122143

123-
ManyToMany Field: `many_to_many_field`
124-
ForeignKey Field: `foreign_key_field_set`
125-
"""
126-
return MyModel.objects.get(id=1)
144+
=== "components.py"
127145

128-
@component
129-
def todo_list():
130-
query = use_query(
131-
io_intensive_operation,
132-
QueryOptions(postprocessor_kwargs={"many_to_many": True, "many_to_one": True}),
133-
)
146+
```python
147+
from example_project.my_app.models import MyModel
148+
from idom import component
149+
from django_idom.types import QueryOptions
150+
from django_idom.hooks import use_query
134151

135-
if query.loading or query.error:
136-
return None
152+
def get_model_with_relationships():
153+
"""This is an example query function that gets `MyModel` which has a ManyToMany field, and
154+
additionally other models that have formed a ForeignKey association to `MyModel`.
137155

138-
return f"{query.data.many_to_many_field} {query.data.foriegn_key_field_set}"
139-
```
156+
ManyToMany Field: `many_to_many_field`
157+
ForeignKey Field: `foreign_key_field_set`
158+
"""
159+
return MyModel.objects.get(id=1)
140160

141-
Please note that due Django's ORM design, the field name to access foreign keys is [always be postfixed with `_set`](https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_one/).
161+
@component
162+
def todo_list():
163+
query = use_query(
164+
get_model_with_relationships,
165+
QueryOptions(postprocessor_kwargs={"many_to_many": False, "many_to_one": False}),
166+
)
167+
168+
if query.loading or query.error:
169+
return None
142170

143-
??? question "Why does the example `get_items` function return a `Model` or `QuerySet`?"
171+
# By disabling `many_to_many` and `many_to_one`, accessing these fields will now
172+
# generate a `SynchronousOnlyOperation` exception
173+
return f"{query.data.many_to_many_field} {query.data.foriegn_key_field_set}"
174+
```
144175

145-
This was a technical design decision to [based on Apollo](https://www.apollographql.com/docs/react/data/mutations/#usemutation-api), but ultimately helps avoid Django's `SynchronousOnlyOperation` exceptions.
176+
_Note: In Django's ORM design, the field name to access foreign keys is [always be postfixed with `_set`](https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_one/)._
146177

147-
The `use_query` hook ensures the provided `Model` or `QuerySet` executes all [deferred](https://docs.djangoproject.com/en/dev/ref/models/instances/#django.db.models.Model.get_deferred_fields)/[lazy queries](https://docs.djangoproject.com/en/dev/topics/db/queries/#querysets-are-lazy) safely prior to reaching your components.
178+
??? question "Can I make ORM calls without hooks?"
179+
180+
{% include-markdown "../../includes/orm.md" start="<!--orm-excp-start-->" end="<!--orm-excp-end-->" %}
148181

149182
## Use Mutation
150183

@@ -202,6 +235,28 @@ The function you provide into this hook will have no return value.
202235
| --- | --- |
203236
| `Mutation[_Params]` | An object containing `loading`/`error` states, a `reset` callable that will set `loading`/`error` states to defaults, and a `execute` callable that will run the query. |
204237

238+
??? question "How can I provide arguments to my mutation function?"
239+
240+
`*args` and `**kwargs` can be provided to your mutation function via `mutation.execute` parameters.
241+
242+
=== "components.py"
243+
244+
```python
245+
from idom import component
246+
from django_idom.hooks import use_mutation
247+
248+
def example_mutation(value:int, other_value:bool = False):
249+
...
250+
251+
@component
252+
def my_component():
253+
mutation = use_mutation(example_mutation)
254+
255+
mutation.execute(123, other_value=True)
256+
257+
...
258+
```
259+
205260
??? question "Can `use_mutation` trigger a refetch of `use_query`?"
206261

207262
Yes, `use_mutation` can queue a refetch of a `use_query` via the `refetch=...` argument.
@@ -225,19 +280,22 @@ The function you provide into this hook will have no return value.
225280

226281
@component
227282
def todo_list():
283+
item_query = use_query(get_items)
284+
item_mutation = use_mutation(add_item, refetch=get_items)
285+
228286
def submit_event(event):
229287
if event["key"] == "Enter":
230288
item_mutation.execute(text=event["target"]["value"])
231289

232-
item_query = use_query(get_items)
290+
# Handle all possible query states
233291
if item_query.loading:
234292
rendered_items = html.h2("Loading...")
235293
elif item_query.error:
236294
rendered_items = html.h2("Error when loading!")
237295
else:
238296
rendered_items = html.ul(html.li(item, key=item) for item in item_query.data)
239297

240-
item_mutation = use_mutation(add_item, refetch=get_items)
298+
# Handle all possible mutation states
241299
if item_mutation.loading:
242300
mutation_status = html.h2("Adding...")
243301
elif item_mutation.error:
@@ -275,14 +333,15 @@ The function you provide into this hook will have no return value.
275333

276334
@component
277335
def todo_list():
336+
item_mutation = use_mutation(add_item)
337+
278338
def reset_event(event):
279339
item_mutation.reset()
280340

281341
def submit_event(event):
282342
if event["key"] == "Enter":
283343
item_mutation.execute(text=event["target"]["value"])
284344

285-
item_mutation = use_mutation(add_item)
286345
if item_mutation.loading:
287346
mutation_status = html.h2("Adding...")
288347
elif item_mutation.error:
@@ -303,11 +362,7 @@ The function you provide into this hook will have no return value.
303362

304363
??? question "Can I make ORM calls without hooks?"
305364

306-
Due to Django's ORM design, database queries must be deferred using hooks. Otherwise, you will see a `SynchronousOnlyOperation` exception.
307-
308-
This may be resolved in a future version of Django containing an asynchronous ORM.
309-
310-
However, even when resolved it is best practice to perform ORM queries within the `use_query` in order to handle `loading` and `error` states.
365+
{% include-markdown "../../includes/orm.md" start="<!--orm-excp-start-->" end="<!--orm-excp-end-->" %}
311366

312367
## Use Websocket
313368

‎src/django_idom/hooks.py

Lines changed: 2 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
from __future__ import annotations
22

33
import asyncio
4-
import contextlib
54
import logging
65
from typing import (
76
Any,
@@ -15,10 +14,6 @@
1514
)
1615

1716
from channels.db import database_sync_to_async as _database_sync_to_async
18-
from django.db.models import ManyToManyField, prefetch_related_objects
19-
from django.db.models.base import Model
20-
from django.db.models.fields.reverse_related import ManyToOneRel
21-
from django.db.models.query import QuerySet
2217
from idom import use_callback, use_ref
2318
from idom.backend.types import Location
2419
from idom.core.hooks import Context, create_context, use_context, use_effect, use_state
@@ -31,7 +26,7 @@
3126
_Params,
3227
_Result,
3328
)
34-
from django_idom.utils import _generate_obj_name
29+
from django_idom.utils import _generate_obj_name, django_query_postprocessor
3530

3631

3732
_logger = logging.getLogger(__name__)
@@ -163,7 +158,7 @@ def execute_query() -> None:
163158

164159
# Use the default postprocessor
165160
else:
166-
postprocess_django_query(data, **query_options.postprocessor_kwargs)
161+
django_query_postprocessor(data, **query_options.postprocessor_kwargs)
167162
except Exception as e:
168163
set_data(None)
169164
set_loading(False)
@@ -233,54 +228,3 @@ def reset() -> None:
233228
set_error(None)
234229

235230
return Mutation(call, loading, error, reset)
236-
237-
238-
def postprocess_django_query(
239-
data: QuerySet | Model, /, many_to_many: bool = False, many_to_one: bool = False
240-
) -> None:
241-
"""Recursively fetch all fields within a `Model` or `QuerySet` to ensure they are not performed lazily.
242-
243-
Some behaviors can be modified through `query_options` attributes."""
244-
245-
# `QuerySet`, which is an iterable of `Model`/`QuerySet` instances
246-
# https://github.com/typeddjango/django-stubs/issues/704
247-
if isinstance(data, QuerySet): # type: ignore[misc]
248-
for model in data:
249-
postprocess_django_query(
250-
model,
251-
many_to_many=many_to_many,
252-
many_to_one=many_to_one,
253-
)
254-
255-
# `Model` instances
256-
elif isinstance(data, Model):
257-
prefetch_fields: list[str] = []
258-
for field in data._meta.get_fields():
259-
# `ForeignKey` relationships will cause an `AttributeError`
260-
# This is handled within the `ManyToOneRel` conditional below.
261-
with contextlib.suppress(AttributeError):
262-
getattr(data, field.name)
263-
264-
if many_to_one and type(field) == ManyToOneRel:
265-
prefetch_fields.append(f"{field.name}_set")
266-
267-
elif many_to_many and isinstance(field, ManyToManyField):
268-
prefetch_fields.append(field.name)
269-
postprocess_django_query(
270-
getattr(data, field.name).get_queryset(),
271-
many_to_many=many_to_many,
272-
many_to_one=many_to_one,
273-
)
274-
275-
if prefetch_fields:
276-
prefetch_related_objects([data], *prefetch_fields)
277-
278-
# Unrecognized type
279-
else:
280-
raise TypeError(
281-
f"Django query postprocessor expected a Model or QuerySet, got {data!r}.\n"
282-
"One of the following may have occurred:\n"
283-
" - You are using a non-Django ORM.\n"
284-
" - You are attempting to use `use_query` to fetch non-ORM data.\n\n"
285-
"If these situations seem correct, you may want to consider disabling the postprocessor via `QueryOptions`."
286-
)

0 commit comments

Comments
 (0)