Zero-config dynamically-generated queryables, Performance fixes

[Zaczero]

Related Issue(s):

• Fixes mk_actions performance issue #345
• Fixes filter_links performance issue #344
• Fixes Replace fastapi-slim with fastapi #336

Description:

This PR consists of self-contained commits (except the first commit that provides database_logic deduplication), making it easy to change or remove individual patches. It addresses several small issues, improves the performance of certain methods, and adds support for dynamically-generated queryables. This enhancement doesn't require any new configuration as queryables are generated on the fly based on the os/es mappings. The implementation is designed for extensibility, with built-in logic for augmenting fields metadata with additional information. Currently, it only includes the _DEFAULT_QUERYABLES configuration, which was simply copied from the pre-PR code.

Example queryables response:

{"$schema":"https://json-schema.org/draft/2019-09/schema","$id":"https://stac-api.example.com/queryables","type":"object","title":"Queryables for STAC API","description":"Queryable names for the STAC API Item Search filter.","properties":{"bbox":{"title":"Bbox","type":"number"},"collection":{"description":"Collection","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json#/definitions/core/allOf/2/then/properties/collection","title":"Collection","type":"string"},"geometry":{"description":"Geometry","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json#/definitions/core/allOf/1/oneOf/0/properties/geometry","title":"Geometry","type":"object"},"id":{"description":"ID","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json#/definitions/core/allOf/2/properties/id","title":"Id","type":"string"},"stac_extensions":{"title":"Stac Extensions","type":"string"},"stac_version":{"title":"Stac Version","type":"string"},"type":{"title":"Type","type":"string"},"constellation":{"title":"Constellation","type":"string"},"created":{"description":"Creation Timestamp","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/datetime.json#/properties/created","title":"Created","type":"string","format":"date-time"},"datetime":{"description":"Acquisition Timestamp","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/datetime.json#/properties/datetime","title":"Datetime","type":"string","format":"date-time"},"end_datetime":{"title":"End Datetime","type":"string","format":"date-time"},"eopf:datatake_id":{"title":"Eopf:Datatake Id","type":"string"},"eopf:instrument_configuration_id":{"title":"Eopf:Instrument Configuration Id","type":"number"},"instruments":{"title":"Instruments","type":"string"},"platform":{"title":"Platform","type":"string"},"processing:datetime":{"title":"Processing:Datetime","type":"string","format":"date-time"},"processing:facility":{"title":"Processing:Facility","type":"string"},"processing:level":{"title":"Processing:Level","type":"string"},"processing:version":{"title":"Processing:Version","type":"string"},"product:timeliness":{"title":"Product:Timeliness","type":"string"},"product:timeliness_category":{"title":"Product:Timeliness Category","type":"string"},"product:type":{"title":"Product:Type","type":"string"},"published":{"title":"Published","type":"string","format":"date-time"},"sar:center_frequency":{"title":"Sar:Center Frequency","type":"number"},"sar:frequency_band":{"title":"Sar:Frequency Band","type":"string"},"sar:instrument_mode":{"title":"Sar:Instrument Mode","type":"string"},"sar:observation_direction":{"title":"Sar:Observation Direction","type":"string"},"sar:pixel_spacing_azimuth":{"title":"Sar:Pixel Spacing Azimuth","type":"number"},"sar:pixel_spacing_range":{"title":"Sar:Pixel Spacing Range","type":"number"},"sar:polarizations":{"title":"Sar:Polarizations","type":"string"},"sar:resolution_azimuth":{"title":"Sar:Resolution Azimuth","type":"number"},"sar:resolution_range":{"title":"Sar:Resolution Range","type":"number"},"sat:absolute_orbit":{"title":"Sat:Absolute Orbit","type":"integer"},"sat:orbit_cycle":{"title":"Sat:Orbit Cycle","type":"number"},"sat:orbit_state":{"title":"Sat:Orbit State","type":"string"},"sat:platform_international_designator":{"title":"Sat:Platform International Designator","type":"string"},"sat:relative_orbit":{"title":"Sat:Relative Orbit","type":"integer"},"start_datetime":{"title":"Start Datetime","type":"string","format":"date-time"},"updated":{"description":"Creation Timestamp","$ref":"https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/datetime.json#/properties/updated","title":"Updated","type":"string","format":"date-time"},"view:azimuth":{"title":"View:Azimuth","type":"number"},"view:incidence_angle":{"title":"View:Incidence Angle","type":"number"},"auth:schemes.oidc.openIdConnectUrl":{"title":"Auth:Schemes.Oidc.Openidconnecturl","type":"string"},"auth:schemes.oidc.type":{"title":"Auth:Schemes.Oidc.Type","type":"string"},"auth:schemes.s3.type":{"title":"Auth:Schemes.S3.Type","type":"string"},"storage:schemes.cdse-s3.description":{"title":"Storage:Schemes.Cdse-S3.Description","type":"string"},"storage:schemes.cdse-s3.platform":{"title":"Storage:Schemes.Cdse-S3.Platform","type":"string"},"storage:schemes.cdse-s3.requester_pays":{"title":"Storage:Schemes.Cdse-S3.Requester Pays","type":"boolean"},"storage:schemes.cdse-s3.title":{"title":"Storage:Schemes.Cdse-S3.Title","type":"string"},"storage:schemes.cdse-s3.type":{"title":"Storage:Schemes.Cdse-S3.Type","type":"string"},"storage:schemes.creodias-s3.description":{"title":"Storage:Schemes.Creodias-S3.Description","type":"string"},"storage:schemes.creodias-s3.platform":{"title":"Storage:Schemes.Creodias-S3.Platform","type":"string"},"storage:schemes.creodias-s3.requester_pays":{"title":"Storage:Schemes.Creodias-S3.Requester Pays","type":"boolean"},"storage:schemes.creodias-s3.title":{"title":"Storage:Schemes.Creodias-S3.Title","type":"string"},"storage:schemes.creodias-s3.type":{"title":"Storage:Schemes.Creodias-S3.Type","type":"string"}},"additionalProperties":false}

PS. I think the auto-generated "title" should be removed completely, but I included it because I found it to be common practice in some STAC projects. I'm not sure how you feel about it.

PR Checklist:

• Code is formatted and linted (run pre-commit run --all-files)
• Tests pass (run make test)
• Documentation has been updated to reflect changes, if applicable
• Changes are added to the changelog

[jonhealy1]

Hi @Zaczero. Looks really good, just a few linting errors.

[jonhealy1]

It would be nice to have @rhysrevans3 look at this as he is doing work on the queryables.

[rhysrevans3]

@jonhealy1 This looks good to me and is probably a neater solution for /queryables than my pull request as it includes the human readable name and doesn't create an extra index.

@Zaczero am I right in saying this doesn't change the filter extension query? So doesn't effect the queryables mapping that is used in to_es

[jonhealy1]

@rhysrevans3 I sent you an invite to be a maintainer of this project so you can do an official pr review.

[rhysrevans3]

These look good!

@jonhealy1 I think my queryables pull request could still be useful. It takes a different approach, extracting the fields from the extensions where possible and does something similar to this pull request for the other properties. I'd be happy to attempt to merge the two methods in my pull request once this one is accepted if we want both?

[jamesfisher-geo]

This looks amazing. I'd like to follow this same approach to generate aggregations as well

[jonhealy1]

Thanks @Zaczero. Looks great!