Reformat transactions, add docs, etc. #45
Conversation
| @@ -1,57 +1,53 @@ | |||
| repos: | |||
There was a problem hiding this comment.
just reformatting
| } | ||
| }, | ||
| # Default all other strings not otherwise specified to keyword | ||
| {"strings": {"match_mapping_type": "string", "mapping": {"type": "keyword"}}}, |
There was a problem hiding this comment.
"strings" is probably the most important one -- this ensures that all string values that aren't otherwise noted (like title and description) are keywords, so they don't get parsed. Without this, you can't do things like sort on id or filter on id when the id has a non-word character (like underscore) in it.
| ] | ||
|
|
||
| mappings = { | ||
| "numeric_detection": False, |
There was a problem hiding this comment.
this turns off the behavior of trying to index numeric-valued strings as numbers, e.g., it maintains that "10" is indexed as the keyword string "10" rather than the number 10.
| "created": {"type": "date"}, | ||
| "updated": {"type": "date"}, | ||
| # Satellite Extension https://github.com/stac-extensions/sat | ||
| "sat:absolute_orbit": {"type": "integer"}, |
There was a problem hiding this comment.
these are both explicitly integral values, so don't let them be floats
| }, | ||
| # Default all other strings not otherwise specified to keyword | ||
| {"strings": {"match_mapping_type": "string", "mapping": {"type": "keyword"}}}, | ||
| {"numerics": {"match_mapping_type": "long", "mapping": {"type": "float"}}}, |
There was a problem hiding this comment.
make numeric values floats instead of integral. Without this, the first value for a field that comes in could be a floating-point that just happens to be an integer, and thereafter all of the values will be truncated. E.g., without this, if the first item had a value for eo:cloud_cover as 10.3, it would be indexed as the long 10, and a query eo:cloud_cover <= 10 would match on it, even though the Item itself would still have 10.3 as the value!
| ignore=400, # ignore 400 already exists code | ||
| ) | ||
|
|
||
| def create_item(self, model: stac_types.Item, **kwargs): | ||
| @overrides |
There was a problem hiding this comment.
renamed the parameters for all of these methods that override ones in the base class to match the names in the base class, and put them in the same order they are in the base class.
| """Bulk item insertion using es.""" | ||
| transactions_client = TransactionsClient() | ||
| transactions_client._create_item_index() | ||
| transactions_client.create_item_index() |
There was a problem hiding this comment.
renamed this method to remove the leading _ because this is a usage outside the class.
| @@ -178,10 +256,13 @@ def bulk_sync(self, processed_items): | |||
| ] | |||
| helpers.bulk(self.client, actions) | |||
|
|
|||
| def bulk_item_insert(self, items: Items, **kwargs) -> str: | |||
| @overrides | |||
| def bulk_item_insert( | |||
There was a problem hiding this comment.
this is the new signature for the method in stac-fastapi.
| index="stac_items", | ||
| body=mapping, | ||
| body={"mappings": self.mappings}, |
There was a problem hiding this comment.
apply the mappings when creating stac_items
philvarner
changed the title
|
I pulled the actual changes to the mappings out of this one, since there were so many other formatting and cleanup changes. |
Related Issue(s):
Description:
PR Checklist:
pre-commit run --all-files)make test)make docs)