Skip to content

Implement Async Schema Registry client #1965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Jun 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4799d92
Refactor for async support
rohitsanj Apr 17, 2025
bd445fd
remove avro module import
rohitsanj Apr 17, 2025
a5fbc78
Add top level __all__ to refactored modules
rohitsanj Apr 21, 2025
e8a2ce9
Merge remote-tracking branch 'origin/master' into refactor-for-async-…
rohitsanj Apr 23, 2025
1e6404d
reduce diffs
rohitsanj Apr 23, 2025
827f14b
refactor
rohitsanj Apr 23, 2025
5cb44dd
formatting
rohitsanj Apr 23, 2025
d4c61f1
Merge branch 'master' into refactor-for-async-support
rohitsanj May 27, 2025
9cf1b08
style fix
rohitsanj May 28, 2025
09d8ebc
fix flake8
rohitsanj May 28, 2025
4ae7877
fix
rohitsanj May 28, 2025
c4cfbcb
fix flake8
rohitsanj May 28, 2025
d2f6b39
Merge branch 'master' into refactor-for-async-support
rohitsanj May 29, 2025
1a6fa96
revert
rohitsanj May 29, 2025
bf64e65
reduce diff
rohitsanj May 29, 2025
95da806
Implement async variants of SR clients and serdes
rohitsanj Apr 17, 2025
32507b5
refactor in prep for adding async
rohitsanj Apr 17, 2025
b1185e0
Implement async variants of SR clients and serdes
rohitsanj Apr 17, 2025
63d385d
Refactor
rohitsanj Apr 21, 2025
f9874ca
make functions async
rohitsanj Apr 23, 2025
2365f8d
make schema id serializer changes to async
rohitsanj May 27, 2025
12b0ec3
fix
rohitsanj May 28, 2025
f9a82b4
add async changes and refactor into separate unasync script
rohitsanj May 28, 2025
1177786
add check to source-package-verification
rohitsanj May 28, 2025
b3ed459
fixes
rohitsanj May 28, 2025
2fb4e3e
fix flake8
rohitsanj May 28, 2025
e9f3544
ignore build
rohitsanj May 28, 2025
09de81c
fix tests
rohitsanj May 29, 2025
7fe4e3f
fix
rohitsanj May 29, 2025
dac2bef
fix flake8
rohitsanj May 29, 2025
a8d05cb
add async sr import
rohitsanj May 30, 2025
02ce930
Auto-generate README
rohitsanj May 30, 2025
05d2918
use timeout
rohitsanj May 30, 2025
187b919
Refactor unasync and add tests to prove functionality
rohitsanj May 30, 2025
5e0d7be
fix
rohitsanj May 30, 2025
3102fed
Merge branch 'master' into add-async-compat-schema-registry
rohitsanj May 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ or:

Documentation will be generated in `build/sphinx/html`.

## Unasync -- maintaining sync versions of async code

$ python tools/unasync.py

# Run the script with the --check flag to ensure the sync code is up to date
$ python tools/unasync.py --check

If you make any changes to the async code (in `src/confluent_kafka/schema_registry/_async` and `tests/integration/schema_registry/_async`), you **must** run this script to generate the sync counter parts (in `src/confluent_kafka/schema_registry/_sync` and `tests/integration/schema_registry/_sync`). Otherwise, this script will be run in CI with the --check flag and fail the build.


## Tests

Expand Down
33 changes: 33 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -652,3 +652,36 @@ For the files wingetopt.c wingetopt.h downloaded from https://github.com/alex85k
*/



LICENSE.unasync
--------------------------------------------------------------
For unasync code in setup.py, derived from
https://github.com/encode/httpcore/blob/ae46dfbd4330eefaa9cd6ab1560dec18a1d0bcb8/scripts/unasync.py

Copyright © 2020, [Encode OSS Ltd](https://www.encode.io/).
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,6 @@ optional-dependencies.all = { file = [
"requirements/requirements-avro.txt",
"requirements/requirements-json.txt",
"requirements/requirements-protobuf.txt"] }

[tool.pytest.ini_options]
asyncio_mode = "auto"
2 changes: 2 additions & 0 deletions requirements/requirements-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ requests-mock
respx
pytest_cov
pluggy<1.6.0
pytest-asyncio
async-timeout
2 changes: 2 additions & 0 deletions src/confluent_kafka/schema_registry/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
RuleSet,
Schema,
SchemaRegistryClient,
AsyncSchemaRegistryClient,
SchemaRegistryError,
SchemaReference,
ServerConfig
Expand All @@ -57,6 +58,7 @@
"RuleSet",
"Schema",
"SchemaRegistryClient",
"AsyncSchemaRegistryClient",
"SchemaRegistryError",
"SchemaReference",
"ServerConfig",
Expand Down
Empty file.
603 changes: 603 additions & 0 deletions src/confluent_kafka/schema_registry/_async/avro.py

Large diffs are not rendered by default.

659 changes: 659 additions & 0 deletions src/confluent_kafka/schema_registry/_async/json_schema.py

Large diffs are not rendered by default.

717 changes: 717 additions & 0 deletions src/confluent_kafka/schema_registry/_async/protobuf.py

Large diffs are not rendered by default.

Loading