Skip to content

Commit e709f32

Browse files
imran-vzNaros
authored andcommitted
fixed md violations and added syntax hightlighted codeblocks
1 parent cd4171a commit e709f32

File tree

1 file changed

+71
-51
lines changed

1 file changed

+71
-51
lines changed

README.md

Lines changed: 71 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -10,59 +10,70 @@
1010
A PostgreSQL logical decoder output plugin to deliver data as [Protocol Buffers](https://developers.google.com/protocol-buffers), adapted for Debezium
1111

1212
## Thanks to
13-
- The original [Decoderbufs Project](https://github.com/xstevens/decoderbufs) on which this is based
13+
14+
- The original [Decoderbufs Project](https://github.com/xstevens/decoderbufs) on which this is based
1415
- [The PostgreSQL Team](https://postgresql.org) for adding [logical decoding](http://www.postgresql.org/docs/9.4/static/logicaldecoding.html) support
1516

1617
## Dependencies
18+
1719
This code depends on the following libraries and requires them for compilation:
1820

19-
* [PostgreSQL](http://www.postgresql.org) 9.6+
20-
* [Protobuf-c](https://github.com/protobuf-c/protobuf-c) 1.2+ - used for data serialization
21-
* [PostGIS](http://www.postgis.net/) 2.1+ - used for Postgres geometric types support
21+
- [PostgreSQL](http://www.postgresql.org) 9.6+
22+
- [Protobuf-c](https://github.com/protobuf-c/protobuf-c) 1.2+ - used for data serialization
23+
- [PostGIS](http://www.postgis.net/) 2.1+ - used for Postgres geometric types support
2224

2325
## Building
2426

2527
`postgres-decoderbufs` has to be built from source after installing required dependencies. The required dependencies are first PostgreSQL
2628
(for pg_config), PostgreSQL server development packages, protobuf-c for the Protocol Buffer support and some PostGIS development packages.
2729

2830
### Installing Dependencies
31+
2932
#### Debian
3033

31-
# Core build utilities
32-
apt-get update && apt-get install -f -y software-properties-common build-essential pkg-config git postgresql-server-dev-9.6
33-
34-
# PostGIS dependency
35-
apt-get install -f -y libproj-dev liblwgeom-dev
34+
```bash
35+
# Core build utilities
36+
apt-get update && apt-get install -f -y software-properties-common build-essential pkg-config git postgresql-server-dev-9.6
3637

37-
# Protobuf-c dependency (requires a non-stable Debian repo)
38-
add-apt-repository "deb http://ftp.debian.org/debian testing main contrib" && apt-get update
39-
apt-get install -y libprotobuf-c-dev=1.2.1-1+b1
38+
# PostGIS dependency
39+
apt-get install -f -y libproj-dev liblwgeom-dev
40+
41+
# Protobuf-c dependency (requires a non-stable Debian repo)
42+
add-apt-repository "deb http://ftp.debian.org/debian testing main contrib" && apt-get update
43+
apt-get install -y libprotobuf-c-dev=1.2.1-1+b1
44+
```
4045

4146
When updating the ProtoBuf definition, also install the ProtoBuf C compiler:
4247

43-
apt-get install -y protobuf-c-compiler=1.2.*
48+
```bash
49+
apt-get install -y protobuf-c-compiler=1.2.*
50+
```
4451

4552
The above are taken from the Debezium [container images](https://github.com/debezium/docker-images).
4653

4754
#### Other Linux distributions
4855

49-
You just need to make sure the above software packages (_or some flavour thereof_) are installed for your distro.
56+
You just need to make sure the above software packages (_or some flavour thereof_) are installed for your distro.
5057
Note that the last step from the above sequence is only required for Debian to be able to install `libprotobuf-c-dev:1.2.1`
5158

5259
### Getting the source code
5360

5461
If you have all of the above prerequisites installed, clone this git repo to build from source:
5562

56-
git clone https://github.com/debezium/postgres-decoderbufs.git
57-
cd postgres-decoderbufs
63+
```bash
64+
git clone https://github.com/debezium/postgres-decoderbufs.git
65+
cd postgres-decoderbufs
66+
```
5867

5968
### Optional: Re-generating ProtoBuf code
6069

6170
This is only needed after changes to the ProtoBuf definition (_proto/pg_logicaldec.proto):
6271

63-
cd proto
64-
protoc-c --c_out=../src/proto pg_logicaldec.proto
65-
cd ..
72+
```bash
73+
cd proto
74+
protoc-c --c_out=../src/proto pg_logicaldec.proto
75+
cd ..
76+
```
6677

6778
Commit the generated files to git then.
6879

@@ -71,49 +82,58 @@ Commit the generated files to git then.
7182
If you have multiple Postgres versions installed, you can select which version to install decoderbufs into by altering your `$PATH` to point to the right version.
7283
Then `make` and `make install` for each version. Here is an example:
7384

74-
# Install for Postgres 9.6 if I have multiple local versions
75-
export PATH=/usr/lib/postgresql/9.6/bin:$PATH
76-
make
77-
make install
85+
```bash
86+
# Install for Postgres 9.6 if I have multiple local versions
87+
export PATH=/usr/lib/postgresql/9.6/bin:$PATH
88+
make
89+
make install
90+
```
7891

7992
Once the extension has been installed you just need to enable it and logical replication in postgresql.conf:
8093

81-
# MODULES
82-
shared_preload_libraries = 'decoderbufs'
83-
84-
# REPLICATION
85-
wal_level = logical # minimal, archive, hot_standby, or logical (change requires restart)
86-
max_wal_senders = 8 # max number of walsender processes (change requires restart)
87-
wal_keep_segments = 4 # in logfile segments, 16MB each; 0 disables
88-
#wal_sender_timeout = 60s # in milliseconds; 0 disables
89-
max_replication_slots = 4 # max number of replication slots (change requires restart)
94+
```bash
95+
# MODULES
96+
shared_preload_libraries = 'decoderbufs'
97+
98+
# REPLICATION
99+
wal_level = logical # minimal, archive, hot_standby, or logical (change requires restart)
100+
max_wal_senders = 8 # max number of walsender processes (change requires restart)
101+
wal_keep_segments = 4 # in logfile segments, 16MB each; 0 disables
102+
#wal_sender_timeout = 60s # in milliseconds; 0 disables
103+
max_replication_slots = 4 # max number of replication slots (change requires restart)
104+
```
90105

91106
In addition, permissions will have to be added for the user that connects to the DB to be able to replicate. This can be modified in _pg\_hba.conf_ like so:
92107

93-
local replication <youruser> trust
94-
host replication <youruser> 127.0.0.1/32 trust
95-
host replication <youruser> ::1/128 trust
96-
108+
```make
109+
local replication <youruser> trust
110+
host replication <youruser> 127.0.0.1/32 trust
111+
host replication <youruser> ::1/128 trust
112+
```
113+
97114
And restart PostgreSQL.
98115

99116
## Usage
100-
-- can use SQL for demo purposes
101-
select * from pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');
102-
103-
-- DO SOME TABLE MODIFICATIONS (see below about UPDATE/DELETE)
104-
105-
-- peek at WAL changes using decoderbufs debug mode for SQL console
106-
select data from pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
107-
-- get WAL changes using decoderbufs to update the WAL position
108-
select data from pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
109-
110-
-- check the WAL position of logical replicators
111-
select * from pg_replication_slots where slot_type = 'logical';
117+
118+
```sql
119+
-- can use SQL for demo purposes
120+
select * from pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');
121+
122+
-- DO SOME TABLE MODIFICATIONS (see below about UPDATE/DELETE)
123+
124+
-- peek at WAL changes using decoderbufs debug mode for SQL console
125+
select data from pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
126+
-- get WAL changes using decoderbufs to update the WAL position
127+
select data from pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
128+
129+
-- check the WAL position of logical replicators
130+
select * from pg_replication_slots where slot_type = 'logical';
131+
```
112132

113133
If you're performing an UPDATE/DELETE on your table and you don't see results for those operations from logical decoding, make sure you have set [REPLICA IDENTITY](http://www.postgresql.org/docs/9.4/static/sql-altertable.html#SQL-CREATETABLE-REPLICA-IDENTITY) appropriately for your use case.
114-
134+
115135
The binary format will be consumed by the Debezium Postgres Connector.
116-
136+
117137
## Type Mappings
118138

119139
The following table shows how current PostgreSQL type OIDs are mapped to which decoderbuf fields:
@@ -144,5 +164,5 @@ The following table shows how current PostgreSQL type OIDs are mapped to which d
144164

145165
## Support
146166

147-
File bug reports and feature requests using [Debezium's JIRA](https://issues.jboss.org/browse/DBZ) and the
167+
File bug reports and feature requests using [Debezium's JIRA](https://issues.jboss.org/browse/DBZ) and the
148168
[postgresql-connector](https://issues.jboss.org/browse/DBZ/component/12323543) component

0 commit comments

Comments
 (0)