Setting Up In Swarm

[stephenstubbs]

Hi,

Firstly I would like to say what an awesome project. I have been trying alternatives and looking at this architecture for some time but haven't been successful in creating a reusable deployment of them in such a clean way as you have done here.

I am trying to test this in a swarm. I am happy to help wherever I can with this project as I think this is something that has been sorely missed in the postgres community.

I am trying to set it up in swarm mode just on one machine for the time being. However only the GTM starts if I start it like docker stack deploy --compose-file docker-compose.image.yml postgresxl

My first thought was that the network local might not work in swarm mode so I changed it to overlay in the docker-compose file like this

networks:
  db_a:
    overlay: true
  db_b:
    overlay: true

however it's still not working. I'm happy to dig into it a bit more and perhaps create a separate yml file for swarm if necessary but perhap is there something simple I'm not doing?

Thanks

[tiredpixel]

Hi, @sstubbs. Thanks a lot for your kind comments.

It should work fine in Swarm (and indeed, Kubernetes, too). How exactly that looks, however, depends on your intended topology, and potentially some stuff about your own setup. It isn't really possible to provide a single file which would work for everyone. With reasonably up-to-date Docker, though, using the modern Docker Swarm (as opposed to the old project), then something like the files already there for docker-compose.yml should work as a basis for a docker stack deploy.

You should be setting overlay: true, indeed; local networking won't flow over the network. Also, I would highly recommend you set internal: true, unless you really, really know you don't want it, to protect your networks somewhat from external access—but it's still critical you double-check pg_hba.conf on all Coordinators and Datanodes after installation, to ensure they're set appropriately; the most up-to-date version I packaged a few weeks ago is far better at autocreating that—but it still needs checking, and maybe changing in some cases to fit your use-case.

Could I please confirm which version of the Docker Hub images you're using? (I presume you're using them, not building the container yourself from scratch?) There's no point in using one older than a few weeks ago, as I vastly improved the healthchecks and networking, whilst still providing an older Postgres-XL version (but with the latest patches installed from upstream).

Also, could I please ask for more information about what exactly fails, and at which stage? Do the images load? Note that you'll need to configure the cluster manually using the instructions in the README, or make your own version of the example bin/init-eg script that fits your topology—note that nodes won't auto-detect, and there's no pgxc_ctl, and all this is on purpose.

I recommend that, if you haven't installed it before, you start with an example using Docker Compose and running the bin/init-eg script; that should create a working cluster. Then, I'd either set up the cluster manually as noted, or adapt the script for your environment (which would be a bit tricky, for Swarm, since you won't be on the same host, and the container names will be different—but it's possible).

[tiredpixel]

p.s. Your starting-point should probably be here:
https://github.com/pavouk-0/postgres-xl-docker/blob/master/docker-compose.image.yml
but with Swarm overlay networking, and changing the latest tag to reference a specific version (you shouldn't use the latest tag in production, but it's fine for testing); note you'll need to ensure those PostgresXL env vars are set somehow (e.g. PG_PORT=6666)—doing so in environment: like in that example file should be fine for Swarm.

[stephenstubbs]

OK thanks a lot. I'll do that and get back to you with what I find. Everything you say makes a lot of sense. I tried both the source and the prebuilt containers.

[stephenstubbs]

Thank you very much. I will try again with what you have said and let you know if I have any issues.

[stephenstubbs]

This issue can be closed. It works perfectly. I will run it in my dev environment for a couple months before I try it in staging. I will let you know once it's all up and running and if I run into any issues.

[tiredpixel]

Excellent; glad to hear it. I would be interested in hearing more about your final setup.