Add more channel adapters

[inputinput77777]

I'd like the project to have more channel adapters and be more flexible out of the box. I'm currently using the Reactive mode, but I think these channel adapters should fit both modes:

• Apache Pulsar
• Apache Solr
• Apache Lucene
• Elasticsearch
• Debezium
• HBase
• Cassandra
• Ignite
• Protobuf (!) / Grpc support in general

[artembilan]

Hi @inputinput77777 !

Thank you for interest in the project!

We are fully aware that there are not enough channel adapters in Spring Integration.
Only the problem that we are short-handed and just don't have resources and time to investigate and implement all the possible use-cases in the world.

The contribution is welcome though: https://github.com/spring-projects/spring-integration/blob/main/CONTRIBUTING.adoc

Probably it is better to start with them in our extensions project: https://github.com/spring-projects/spring-integration-extensions.

You can find there some initial support for Apache Cassandra though: https://github.com/spring-projects/spring-integration-extensions/tree/main/spring-integration-cassandra.

The GRPC discussion you can find here: #2626

Some Elasticsearch is implemented as Spring Cloud Stream Sink application: https://github.com/spring-cloud/stream-applications/tree/main/applications/sink/elasticsearch-sink.
A Debezium support is represented there as a Source: https://github.com/spring-cloud/stream-applications/tree/main/applications/source/cdc-debezium-source.

Probably there are some community efforts in the Internet for others as well, e.g. I see this Spring Boot support for Apache Pulsar: https://github.com/majusko/pulsar-java-spring-boot-starter.

So, in short we are open for supporting more protocols and systems only the problem when our team would be able to dedicate the time for their implementations.
Therefore my ask about contribution.

Thank you again!

[artembilan]

The Elasticsearch is discussed here: #3527

[almogtavor]

Seems pretty messy and hard to follow when there are so many half solutions like that. I think there's should be an issue that will track the creation & status of 10-20 new channel adapters. I also wonder if the repos @artembilan have mentioned are relevant. Would you add to SI a channel adapter that is based on a private community contributor's repo?

[artembilan]

Sure! We can have a discussion with the team.
But again: the support and releasing boundaries bring their effect how we do things over here.
That is not correct to judge only by the number of modules in the the project.
See the table in the docs: https://docs.spring.io/spring-integration/docs/current/reference/html/endpoint-summary.html#spring-integration-endpoints.
And those only from the official main project.

From now let's stay with our current rules: if you want to contribute new module, start from the extensions repo.

[almogtavor]

I personally think it better for the components to be in the same repo, at least for the veteran ones that exists for some months and no bugs found. But what you prefer is what you prefer. Why do you think it's wrong to judge an integration repo based on the number of components? This absolutely is a parameter (of course there are many more, but a user won't care how great is the Redis component of he doesn't use Redis). That's why I personally think it should get more focus.

[artembilan]

One module in Spring Integration may have many channel adapters. For example mail module has support for all the mail protocols. The "component" in our case is some class(es) correspond(s) to some specific functionality. It does not justify to extract them into a separate module (jar). If you pay closer attention to core module, you'll find many components which implements these or those IEP. but still: they are in this single module (jar) - not in the individual managable artifact.

[almogtavor]

You right and that is really great. Also the jdbc support. I talk more about the NoSQL databases and other data products (such as Pulsar etc) that doesn't have support, or that have a hidden support (that can be found only after more digging - such as for Elasticsearch/ Cassandra and more)

[artembilan]

Sure! That's why I said that I will bring this to the team attention. You gave me a lot of food for thinking.
I'll come back to with some decision later on.
But now extensions is the good place to start contributing.

Thank you again!

[artembilan]

Another similar request: spring-projects/spring-integration-extensions#254