Skip to content

bugfix: correct indentation of additionalProperties that are union types #266 #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
ModelWithAdditionalPropertiesInlinedAdditionalProperty,
)
from .model_with_additional_properties_refed import ModelWithAdditionalPropertiesRefed
from .model_with_any_json_properties import ModelWithAnyJsonProperties
from .model_with_any_json_properties_additional_property import ModelWithAnyJsonPropertiesAdditionalProperty
from .model_with_primitive_additional_properties import ModelWithPrimitiveAdditionalProperties
from .model_with_primitive_additional_properties_a_date_holder import ModelWithPrimitiveAdditionalPropertiesADateHolder
from .model_with_union_property import ModelWithUnionProperty
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
from typing import Any, Dict, List, Union, cast

import attr

from ..models.model_with_any_json_properties_additional_property import ModelWithAnyJsonPropertiesAdditionalProperty
from ..types import Unset


@attr.s(auto_attribs=True)
class ModelWithAnyJsonProperties:
""" """

additional_properties: Dict[
str, Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]
] = attr.ib(init=False, factory=dict)

def to_dict(self) -> Dict[str, Any]:

field_dict: Dict[str, Any] = {}
for prop_name, prop in self.additional_properties.items():
if isinstance(prop, ModelWithAnyJsonPropertiesAdditionalProperty):
field_dict[prop_name] = prop.to_dict()

elif isinstance(prop, list):
field_dict[prop_name] = prop

elif isinstance(prop, str):
field_dict[prop_name] = prop
elif isinstance(prop, float):
field_dict[prop_name] = prop
elif isinstance(prop, int):
field_dict[prop_name] = prop
else:
field_dict[prop_name] = prop

field_dict.update({})

return field_dict

@staticmethod
def from_dict(src_dict: Dict[str, Any]) -> "ModelWithAnyJsonProperties":
d = src_dict.copy()
model_with_any_json_properties = ModelWithAnyJsonProperties()

additional_properties = {}
for prop_name, prop_dict in d.items():

def _parse_additional_property(
data: Any,
) -> Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]:
data = None if isinstance(data, Unset) else data
additional_property: Union[
ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool
]
try:
additional_property = ModelWithAnyJsonPropertiesAdditionalProperty.from_dict(data)

return additional_property
except: # noqa: E722
pass
try:
additional_property = cast(List[str], data)

return additional_property
except: # noqa: E722
pass
return cast(str, prop_dict)
return cast(float, prop_dict)
return cast(int, prop_dict)
return cast(bool, prop_dict)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this is ugly. I'll see if I can find the cause and propose a solution further down.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would do this before, just without the casts. It's because in union_property.pyi we loop through each property and if it doesn't have a template we just return it.

What is here is ugly, yes, but it works. However, I do think there is an issue if we declare a templated type before a primitive type - then the primitive type would cause it to return early. Something , like

try:
    additional_property = cast(List[str], data)

    return additional_property
except:  # noqa: E722
    pass
return cast(str, prop_dict)
try:
    additional_property = ModelWithAnyJsonPropertiesAdditionalProperty.from_dict(data)

    return additional_property
except:  # noqa: E722
    pass
return cast(float, data)
return cast(int, data)
return cast(bool, data)

So I think we need to loop over templated properties first, then untemplated properties. Also we'll want to figure out lists, as casting it to a List will not error so that breaks the try/catch logic.


While responding to this comment, I realized that these should be using cast(<primitive type>, data) not cast(<primitive type>, prop_dict) - I added commits to this branch to change that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try looking at this soon as I have time


additional_property = _parse_additional_property(prop_dict)

additional_properties[prop_name] = additional_property

model_with_any_json_properties.additional_properties = additional_properties
return model_with_any_json_properties

@property
def additional_keys(self) -> List[str]:
return list(self.additional_properties.keys())

def __getitem__(
self, key: str
) -> Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]:
return self.additional_properties[key]

def __setitem__(
self, key: str, value: Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]
) -> None:
self.additional_properties[key] = value

def __delitem__(self, key: str) -> None:
del self.additional_properties[key]

def __contains__(self, key: str) -> bool:
return key in self.additional_properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from typing import Any, Dict, List

import attr


@attr.s(auto_attribs=True)
class ModelWithAnyJsonPropertiesAdditionalProperty:
""" """

additional_properties: Dict[str, str] = attr.ib(init=False, factory=dict)

def to_dict(self) -> Dict[str, Any]:

field_dict: Dict[str, Any] = {}
field_dict.update(self.additional_properties)
field_dict.update({})

return field_dict

@staticmethod
def from_dict(src_dict: Dict[str, Any]) -> "ModelWithAnyJsonPropertiesAdditionalProperty":
d = src_dict.copy()
model_with_any_json_properties_additional_property = ModelWithAnyJsonPropertiesAdditionalProperty()

model_with_any_json_properties_additional_property.additional_properties = d
return model_with_any_json_properties_additional_property

@property
def additional_keys(self) -> List[str]:
return list(self.additional_properties.keys())

def __getitem__(self, key: str) -> str:
return self.additional_properties[key]

def __setitem__(self, key: str, value: str) -> None:
self.additional_properties[key] = value

def __delitem__(self, key: str) -> None:
del self.additional_properties[key]

def __contains__(self, key: str) -> bool:
return key in self.additional_properties
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
ModelWithAdditionalPropertiesInlinedAdditionalProperty,
)
from .model_with_additional_properties_refed import ModelWithAdditionalPropertiesRefed
from .model_with_any_json_properties import ModelWithAnyJsonProperties
from .model_with_any_json_properties_additional_property import ModelWithAnyJsonPropertiesAdditionalProperty
from .model_with_primitive_additional_properties import ModelWithPrimitiveAdditionalProperties
from .model_with_primitive_additional_properties_a_date_holder import ModelWithPrimitiveAdditionalPropertiesADateHolder
from .model_with_union_property import ModelWithUnionProperty
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
from typing import Any, Dict, List, Union, cast

import attr

from ..models.model_with_any_json_properties_additional_property import ModelWithAnyJsonPropertiesAdditionalProperty
from ..types import Unset


@attr.s(auto_attribs=True)
class ModelWithAnyJsonProperties:
""" """

additional_properties: Dict[
str, Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]
] = attr.ib(init=False, factory=dict)

def to_dict(self) -> Dict[str, Any]:

field_dict: Dict[str, Any] = {}
for prop_name, prop in self.additional_properties.items():
if isinstance(prop, ModelWithAnyJsonPropertiesAdditionalProperty):
field_dict[prop_name] = prop.to_dict()

elif isinstance(prop, list):
field_dict[prop_name] = prop

elif isinstance(prop, str):
field_dict[prop_name] = prop
elif isinstance(prop, float):
field_dict[prop_name] = prop
elif isinstance(prop, int):
field_dict[prop_name] = prop
else:
field_dict[prop_name] = prop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like maybe we need a way to indicate whether or not a given property actually needs to be transformed and if not, just have an else. Then something similar with construct for from_dict.


field_dict.update({})

return field_dict

@staticmethod
def from_dict(src_dict: Dict[str, Any]) -> "ModelWithAnyJsonProperties":
d = src_dict.copy()
model_with_any_json_properties = ModelWithAnyJsonProperties()

additional_properties = {}
for prop_name, prop_dict in d.items():

def _parse_additional_property(
data: Any,
) -> Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]:
data = None if isinstance(data, Unset) else data
additional_property: Union[
ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool
]
try:
additional_property = ModelWithAnyJsonPropertiesAdditionalProperty.from_dict(data)

return additional_property
except: # noqa: E722
pass
try:
additional_property = cast(List[str], data)

return additional_property
except: # noqa: E722
pass
return cast(str, prop_dict)
return cast(float, prop_dict)
return cast(int, prop_dict)
return cast(bool, prop_dict)

additional_property = _parse_additional_property(prop_dict)

additional_properties[prop_name] = additional_property

model_with_any_json_properties.additional_properties = additional_properties
return model_with_any_json_properties

@property
def additional_keys(self) -> List[str]:
return list(self.additional_properties.keys())

def __getitem__(
self, key: str
) -> Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]:
return self.additional_properties[key]

def __setitem__(
self, key: str, value: Union[ModelWithAnyJsonPropertiesAdditionalProperty, List[str], str, float, int, bool]
) -> None:
self.additional_properties[key] = value

def __delitem__(self, key: str) -> None:
del self.additional_properties[key]

def __contains__(self, key: str) -> bool:
return key in self.additional_properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from typing import Any, Dict, List

import attr


@attr.s(auto_attribs=True)
class ModelWithAnyJsonPropertiesAdditionalProperty:
""" """

additional_properties: Dict[str, str] = attr.ib(init=False, factory=dict)

def to_dict(self) -> Dict[str, Any]:

field_dict: Dict[str, Any] = {}
field_dict.update(self.additional_properties)
field_dict.update({})

return field_dict

@staticmethod
def from_dict(src_dict: Dict[str, Any]) -> "ModelWithAnyJsonPropertiesAdditionalProperty":
d = src_dict.copy()
model_with_any_json_properties_additional_property = ModelWithAnyJsonPropertiesAdditionalProperty()

model_with_any_json_properties_additional_property.additional_properties = d
return model_with_any_json_properties_additional_property

@property
def additional_keys(self) -> List[str]:
return list(self.additional_properties.keys())

def __getitem__(self, key: str) -> str:
return self.additional_properties[key]

def __setitem__(self, key: str, value: str) -> None:
self.additional_properties[key] = value

def __delitem__(self, key: str) -> None:
del self.additional_properties[key]

def __contains__(self, key: str) -> bool:
return key in self.additional_properties
32 changes: 32 additions & 0 deletions end_to_end_tests/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -806,6 +806,38 @@
"additionalProperties": {
"$ref": "#/components/schemas/AnEnum"
}
},
"ModelWithAnyJsonProperties": {
"title": "ModelWithAnyJsonProperties",
"type": "object",
"additionalProperties": {
"anyOf": [
{
"type": "object",
"additionalProperties": {
"type": "string"
}
},
{
"type": "array",
"items": {
"type": "string"
}
},
{
"type": "string"
},
{
"type": "number"
},
{
"type": "integer"
},
{
"type": "boolean"
}
]
}
}
}
}
Expand Down
4 changes: 4 additions & 0 deletions openapi_python_client/parser/properties/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,10 @@ def get_type_string(self, no_optional: bool = False) -> str:
type_string = f"Union[Unset, {type_string}]"
return type_string

def get_instance_type_string(self) -> str:
"""Get a string representation of runtime type that should be used for `isinstance` checks"""
return "list"

def get_imports(self, *, prefix: str) -> Set[str]:
"""
Get a set of import strings that should be included when this property is used somewhere
Expand Down
4 changes: 4 additions & 0 deletions openapi_python_client/parser/properties/property.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ def get_type_string(self, no_optional: bool = False) -> str:
type_string = f"Union[Unset, {type_string}]"
return type_string

def get_instance_type_string(self) -> str:
"""Get a string representation of runtime type that should be used for `isinstance` checks"""
return self.get_type_string(no_optional=True)

# noinspection PyUnusedLocal
def get_imports(self, *, prefix: str) -> Set[str]:
"""
Expand Down
2 changes: 1 addition & 1 deletion openapi_python_client/templates/model.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ class {{ model.reference.class_name }}:
{% if model.additional_properties.template %}
{% from "property_templates/" + model.additional_properties.template import transform %}
for prop_name, prop in self.additional_properties.items():
{{ transform(model.additional_properties, "prop", "field_dict[prop_name]") | indent(4) }}
{{ transform(model.additional_properties, "prop", "field_dict[prop_name]") | indent(12) }}
{% else %}
field_dict.update(self.additional_properties)
{% endif %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def _parse_{{ property.python_name }}(data: Any) -> {{ property.get_type_string(
{{ construct(inner_property, "data", initial_value="UNSET") | indent(4) }}
return {{ property.python_name }}
{% else %}
return {{ source }}
return cast({{ inner_property.get_type_string() }}, {{ source }})
{% endif %}
{% endfor %}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if we aren't running a construct macro because the inner prop doesn't have a template, then we really aren't doing anything with it. So instead of having this in a loop, we could just do this after the loop:

return cast({{ property.get_type_string() }}, data)

So basically, loop through all the things that require some construction, if none of those work, just return what we have and assume it's one of the values. Technically it's one of the values we didn't check yet, but that doesn't really matter to the return type since the calling function doesn't know which union member was selected.

Also, it doesn't look like we actually need to use source anymore since the param is data right? So we could pull that out of the macro.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do still need source because it's used in the outer function to call this inner one (just below on line 22, right before the endmacro)


Expand All @@ -39,9 +39,9 @@ elif {{ source }} is None:
{% endif %}
{% for inner_property in property.inner_properties %}
{% if loop.first and property.required and not property.nullable %}{# No if UNSET or if None statement before this #}
if isinstance({{ source }}, {{ inner_property.get_type_string(no_optional=True) }}):
if isinstance({{ source }}, {{ inner_property.get_instance_type_string() }}):
{% elif not loop.last %}
elif isinstance({{ source }}, {{ inner_property.get_type_string(no_optional=True) }}):
elif isinstance({{ source }}, {{ inner_property.get_instance_type_string() }}):
{% else %}
else:
{% endif %}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar story here I think, we just change the order of some checks. Something like

{% for inner_property in (inner_property for inner_property in property.inner_properties if inner_property.template) %}
{% if loop.first and property.required and not property.nullable %}{# No if UNSET or if None statement before this #}
if isinstance({{ source }}, {{ inner_property.get_instance_type_string() }}):
{% else %}
elif isinstance({{ source }}, {{ inner_property.get_instance_type_string() }}):
{% from "property_templates/" + inner_property.template import transform %}
    {{ transform(inner_property, source, destination, declare_type=False) | indent(4) }}
{% endif %}
{% endfor %}
else:
    {{ destination }} = {{ source }}

Editing in GH UI, so probably some mistakes made, but hopefully you get the idea.

Expand Down