Skip to content

Commit 0e58bf0

Browse files
cuprumtancuprumtan
committed
Added version 3.0.0:
- Template name changed from PostgresPro-[PLATFORM] to Mamonsu PostgreSQL [PLATFORM]; - Changed PostgreSQL uptime metric type to unixtime (date and time of service start); - Changed Cache Hit Ratio metric calculation: now it evaluates via Zabbix calculated item; - Changed Archive Command plugin metrics: now they evaluate without accessing the FS; - Metric name pgsql.transactions [total] changed to pgsql.transactions [committed]; - Added new PG14 metrics based on pg_stat_wal and pg_stat_statements_info; - Removed graphs representing only one metric (since this duplicates the functionality of Zabbix simple graphs); - Added new graphs: PostgreSQL bgwriter buffers, PostgreSQL bgwriter write/sync, PostgreSQL checkpoint count, PostgreSQL checkpoint write/sync; - Added screens: Overview, PostgreSQL Instance, PostgreSQL WAL, PostgreSQL Locks, PostgreSQL Transactions, System; - Fixed support for different Zabbix versions (3.0-5.4): - Added new bootstrap flag x/--create-extensions to optionally create extensions; - Added default mamonsu user creation to bootstrap tool; - Added mamonsu own schema creation to bootrtap tool; - Changed mamonsu plugin structure: added new parameter for Zabbix screens; - Fixed #159;
1 parent 18e03ee commit 0e58bf0

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+7920
-1082
lines changed

‎README.md

Lines changed: 364 additions & 0 deletions
Large diffs are not rendered by default.

‎README.rst

Lines changed: 0 additions & 522 deletions
This file was deleted.

‎documentation/adding_custom_plugins.md

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Mamonsu: adding custom plugins
2+
3+
You can extend Mamonsu with your own custom plugins, as follows:
4+
5+
1. Save all custom plugins in a single directory, such as */etc/mamonsu/plugins*.
6+
2. Make sure this directory is specified in your configuration file under the [plugins] section:
7+
```editorconfig
8+
[plugins]
9+
directory = /etc/mamonsu/plugins
10+
```
11+
3. Generate a new Zabbix template to include custom plugins:
12+
```shell
13+
mamonsu export template template.xml --add-plugins=/etc/mamonsu/plugins
14+
```
15+
4. Upload the generated template.xml to the Zabbix server as explained in sections “[Installation](../README.md#installation)" and "[Configuration](../README.md#configuration)”.
16+
17+
***
18+
19+
## How to write custom plugin
20+
21+
Plugins have a fixed structure:
22+
```python
23+
# all possible elements are listed here, but they are optional and it is not necessary to add all of them
24+
25+
# -*- coding: utf-8 -*-
26+
27+
from mamonsu.plugins.pgsql.plugin import PgsqlPlugin as Plugin
28+
from .pool import Pooler
29+
from mamonsu.lib.zbx_template import ZbxTemplate
30+
31+
32+
class PLUGIN_NAME(Plugin):
33+
# metrics collection interval in seconds
34+
Interval = PLUGIN INTERVAL
35+
36+
# Plugin type specifies which group new metrics will belong to. Plugin type can be one of 'pg', 'sys' or 'all:
37+
# 'pg' configures PostgreSQL metrics
38+
# 'sys' configures system metrics
39+
# 'all' configures both PostgreSQL and system metrics
40+
AgentPluginType = 'PLUGIN TYPE'
41+
42+
# queries to form sql files
43+
query_plugin_metric_for_sql_file = 'QUERY'
44+
# example:
45+
# query_bloating_tables = "select count(*) from pg_catalog.pg_stat_all_tables where " \
46+
# "(n_dead_tup/(n_live_tup+n_dead_tup)::float8) > {0} and " \
47+
# "(n_live_tup+n_dead_tup) > {1};"
48+
49+
# queries for zabbix agent
50+
query_plugin_metric_for_zabbix_agent = 'QUERY'
51+
# example:
52+
# query_size = "select pg_database_size(datname::text) from pg_catalog.pg_database where" \
53+
# " datistemplate = false and datname = :'p1';"
54+
55+
# zabbix elements keys
56+
key_plugin_metric = 'KEY'
57+
# example:
58+
# key_db_bloating_tables = 'pgsql.database.bloating_tables{0}'
59+
60+
# plugin options
61+
DEFAULT_CONFIG = {OPTIONS}
62+
# example:
63+
# DEFAULT_CONFIG = {'min_rows': str(50), 'bloat_scale': str(0.2)}
64+
65+
# run(self, zbx) specifies queries execution and processing
66+
def run(self, zbx):
67+
result = Pooler.query(self.query_plugin_metric_for_sql_file)
68+
zbx.send(self.key_plugin_metric, int(result[0][0]))
69+
70+
# ====================================================================================
71+
# Methods for generating Zabbix objects
72+
# Each method has fixed signature (self, template, dashboard),
73+
# where 'dashboard' parameter specifies what type of object we are going to generate:
74+
# False for generating template objects;
75+
# True for adding objects to screen.
76+
def items(self, template, dashboard=False):
77+
if not dashboard:
78+
result = template.item({
79+
'name': 'ITEM NAME',
80+
'key': self.right_type(self.key_plugin_metric), # right_type() generates key depending on template type - mamonsu template key or zabbix agent template key
81+
'delay': self.plugin_config('interval'),
82+
'value_type': VALUE TYPE, # see available values in mamonsu/lib/const.py VALUE_TYPE
83+
'units': UNITS, # see available values in mamonsu/lib/const.py UNITS
84+
'delta': DELTA # see available values in mamonsu/lib/const.py DELTA
85+
})
86+
return result
87+
else:
88+
# to add item (as simple graph) to Zabbix screen we must define:
89+
# item key ('name')
90+
# screen name ('page')
91+
# graph size ('size')
92+
# position on screen ('position', but it's optionally - you can set 0 to avoid sorting)
93+
return [{'dashboard': {'name': self.right_type(self.key_autovacumm),
94+
'page': ZbxTemplate.dashboard_page_PAGE['name'], # see available values in mamonsu/lib/zbx_template.py dashboard_page_*
95+
'size': ZbxTemplate.dashboard_widget_size_SIZE, # see available values in mamonsu/lib/zbx_template.py dashboard_widget_size_*
96+
'position': N}}]
97+
98+
def discovery_rules(self, template, dashboard=False):
99+
rule = {
100+
'name': 'DISCOVERY RULE NAME',
101+
'key': self.key_discovery_key
102+
}
103+
items = [{
104+
'name': 'ITEM NAME',
105+
'key': self.right_type(self.key_plugin_metric),
106+
'delay': self.plugin_config('interval'),
107+
'value_type': VALUE TYPE,
108+
'units': UNITS,
109+
'delta': DELTA
110+
}]
111+
graphs = [{
112+
'name': 'GRAPH NAME',
113+
'type': GRAPH TYPE,
114+
'items': [{
115+
'name': 'ITEM NAME',
116+
'key': self.right_type(self.key_plugin_metric),
117+
'delay': self.plugin_config('interval'),
118+
'value_type': VALUE TYPE,
119+
'units': UNITS,
120+
'delta': DELTA
121+
}]
122+
}]
123+
return template.discovery_rule(rule=rule, conditions=conditions, items=items, graphs=graphs)
124+
125+
def graphs(self, template, dashboard=False):
126+
result = template.graph({'name': 'GRAPH NAME',
127+
'items': {
128+
'name': 'ITEM NAME',
129+
'key': self.right_type(self.key_plugin_metric),
130+
'delay': self.plugin_config('interval'),
131+
'value_type': VALUE TYPE,
132+
'units': UNITS,
133+
'delta': DELTA
134+
}})
135+
if not dashboard:
136+
return result
137+
else:
138+
# to add graph to Zabbix screen we must define:
139+
# graph name ('name')
140+
# screen name ('page')
141+
# graph size ('size')
142+
# position on screen ('position', but it's optionally - you can set 0 to avoid sorting)
143+
return [{'dashboard': {'name': 'GRAPH NAME',
144+
'page': ZbxTemplate.dashboard_page_PAGE['name'], # see available values in mamonsu/lib/zbx_template.py dashboard_page_*
145+
'size': ZbxTemplate.dashboard_widget_size_SIZE, # see available values in mamonsu/lib/zbx_template.py dashboard_widget_size_*
146+
'position': N}}]
147+
148+
def triggers(self, template, dashboard=False):
149+
result = template.trigger({
150+
'name': 'TRIGGER NAME',
151+
'expression': 'TRIGGER EXPRESSION'
152+
})
153+
154+
return result
155+
156+
# ====================================================================================
157+
158+
# keys_and_queries(self, template_zabbix) generates parameters files for Zabbix agent (by command export zabbix-parameters)
159+
def keys_and_queries(self, template_zabbix):
160+
result = ['{0},$2 $1 -c "{1}"'.format(self.key_plugin_metric, self.query_plugin_metric_for_zabbix_agent)]
161+
return template_zabbix.key_and_query(result)
162+
```
163+
164+
Native Mamonsu plugins are stored in [mamonsu/plugins](../mamonsu/plugins).

‎documentation/configuration_file.md

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
# Mamonsu: configuration file
2+
3+
## Configuration Parameters
4+
The agent.conf configuration file is located in the /etc/mamonsu directory by default. It provides several groups of parameters that control which metrics to collect and how to log the collected data:
5+
- [Connection Parameters](#connection-parameters);
6+
- [Logging parameters](#logging-parameters);
7+
- [Plugin Parameters](#plugin-parameters).
8+
9+
All parameters must be specified in the parameter = value format.
10+
11+
***
12+
13+
### Connection Parameters
14+
**[postgres]**
15+
The [postgres] section controls PostgreSQL metrics collection and can contain the following parameters:
16+
17+
**enabled**
18+
        Enables/disables PostgreSQL metrics collection if set to True or False, respectively.
19+
20+
        Default: True
21+
22+
**user**
23+
        The user on behalf of which the cluster will be monitored. It must be the same user that you specified in the -M option of the bootstrap command, or a superuser if you skipped the bootstrap.
24+
25+
        Default: postgres
26+
27+
**password**
28+
        The password for the specified user.
29+
30+
**database**
31+
        The database to connect to for metrics collection.
32+
33+
        Default: postgres
34+
35+
**host**
36+
        The server address to connect to.
37+
38+
        Default: localhost
39+
40+
**port**
41+
        The port to connect to.
42+
43+
        Default: 5432
44+
45+
**application_name**
46+
        Application name that identifies Mamonsu connected to the PostgreSQL cluster.
47+
48+
        Default: mamonsu
49+
50+
**query_timeout**
51+
        [statement_timeout](https://postgrespro.com/docs/postgresql/13/runtime-config-client#GUC-STATEMENT-TIMEOUT) for the Mamonsu session, in seconds. If a PostgreSQL metric query does not complete within this time interval, it gets terminated.
52+
53+
        Default: 10
54+
55+
<p>&nbsp;</p>
56+
57+
**[system]**
58+
The [system] section controls system metrics collection and can contain the following parameters:
59+
60+
**enabled**
61+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enables/disables system metrics collection if set to True or False, respectively.
62+
63+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: True
64+
65+
<p>&nbsp;</p>
66+
67+
**[zabbix]**
68+
The [zabbix] section provides connection settings for the Zabbix server and can contain the following parameters:
69+
70+
**enabled**
71+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enables/disables sending the collected metric data to the Zabbix server if set to True or False, respectively.
72+
73+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: True
74+
75+
**client**
76+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the Zabbix host.
77+
78+
**address**
79+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The address of the Zabbix server.
80+
81+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 127.0.0.1
82+
83+
**port**
84+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The port of the Zabbix server.
85+
86+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 10051
87+
88+
<p>&nbsp;</p>
89+
90+
**[agent]**
91+
The [agent] section specifies the location of Mamonsu and whether it is allowed to access metrics from the command line:
92+
93+
**enabled**
94+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enables/disables metrics collection from the command line using the agent command.
95+
96+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: True
97+
98+
**host**
99+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The address of the system on which Mamonsu is running.
100+
101+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 127.0.0.1
102+
103+
**port**
104+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The port on which Mamonsu is running.
105+
106+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 10052
107+
108+
<p>&nbsp;</p>
109+
110+
**[sender]**
111+
The [sender] section controls the queue size of the data to be sent to the Zabbix server:
112+
113+
**queue**
114+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The maximum number of collected metric values that can be accumulated locally before mamonsu sends them to the Zabbix server. Once the accumulated data is sent, the queue is cleared.
115+
116+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 2048
117+
118+
***
119+
120+
### Logging Parameters
121+
**[metric_log]**
122+
The [metric_log] section enables storing the collected metric data in text files locally. This section can contain the following parameters:
123+
124+
**enabled**
125+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enables/disables storing the collected metric data in a text file. If this option is set to True, Mamonsu creates the localhost.log file for storing metric values.
126+
127+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: False
128+
129+
**directory**
130+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specifies the directory where log files with metric data will be stored.
131+
132+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: /var/log/mamonsu
133+
134+
**max_size_mb**
135+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The maximum size of a log file, in MB. When the specified size is reached, it is renamed to localhost.log.archive, and an empty localhost.log file is created.
136+
137+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 1024
138+
139+
<p>&nbsp;</p>
140+
141+
**[log]**
142+
The [log] section specifies logging settings for Mamonsu and can contain the following parameters:
143+
144+
**file**
145+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specifies the log filename, which can be preceded by the full path.
146+
147+
**level**
148+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specifies the debug level. This option can take DEBUG, ERROR, or INFO values.
149+
150+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: INFO
151+
152+
**format**
153+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The format of the logged data.
154+
155+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: [%(levelname)s] %(asctime)s - %(name)s - %(message)s
156+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;where levelname is the debug level, asctime returns the current time, name specifies the plugin that emitted this log entry or root otherwise, and message provides the actual log message.
157+
158+
***
159+
160+
### Plugin Parameters
161+
**[plugins]**
162+
The [plugins] section specifies custom plugins to be added for metrics collection and can contain the following parameters:
163+
164+
**enabled**
165+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enables/disables using custom plugins for metrics collection if set to True or False, respectively.
166+
167+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: False
168+
169+
**directory**
170+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specifies the directory that contains custom plugins for metrics collection. Setting this parameter to None forbids using custom plugins.
171+
172+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: /etc/mamonsu/plugins
173+
174+
If you need to configure any of the plugins you add to Mamonsu after installation, you have to add this plugin section to the agent.conf file.
175+
176+
The syntax of this section should follow the syntax used with the examples shown below in the section called “[Individual Plugin Sections](#individual-plugin-sections)”.
177+
178+
***
179+
180+
### Individual Plugin Sections
181+
All built-in plugins are installed along with Mamonsu. To configure a built-in plugin you should find a corresponding section below the Individual Plugin Sections heading and edit its parameter values.
182+
183+
To disable any plugin you should set the enabled parameter to False and to enable it — set it to True. These values are case sensitive.
184+
185+
The example below shows individual plugin sections corresponding to the *preparedtransaction* and the *pgprobackup* built-in plugins:
186+
```editorconfig
187+
[preparedtransaction]
188+
max_prepared_transaction_time = 60
189+
interval = 60
190+
191+
[pgprobackup]
192+
enabled = false
193+
interval = 300
194+
backup_dirs = /backup_dir1,/backup_dir2
195+
pg_probackup_path = /usr/bin/pg_probackup-11
196+
```
197+
198+
**[preparedtransaction]**
199+
This plugin gets age in seconds of the oldest prepared transaction and number of all transactions prepared for a two-phase commit. For additional information refer to [PREPARE TRANSACTION](https://postgrespro.com/docs/postgresql/13/sql-prepare-transaction) and [pg_prepared_xacts](https://postgrespro.com/docs/postgresql/13/view-pg-prepared-xacts).
200+
201+
The *max_prepared_transaction_time parameter* specifies the threshold in seconds for the age of the prepared transaction.
202+
203+
The interval parameter allows you to change the metrics collection interval.
204+
205+
The plugin collects two metrics: *pgsql.prepared.count* (number of prepared transactions) and *pgsql.prepared.oldest* (oldest prepared transaction age in seconds).
206+
207+
If *pgsql.prepared.oldest* value exceeds the threshold specified by the *max_prepared_transaction_time* parameter, a trigger with the following message is fired: "PostgreSQL prepared transaction is too old on {host}".
208+
209+
**[pgprobackup]**
210+
This plugin uses pg_probackup to track its backups' state and gets size of backup directories which store all backup files.
211+
212+
Please note that this plugin counts the total size of all files in the target directory. Make sure that extraneous files are not stored in this directory.
213+
214+
The *backup_dirs* parameter specifies a comma-separated list of paths to directories for which metrics should be collected.
215+
216+
The *pg_probackup_path* parameter specifies the path to pg_probackup.
217+
218+
The *interval* parameter allows you to change the metrics collection interval.
219+
220+
By default this plugin is disabled. To enable it set the enabled parameter to True.
221+
222+
This plugin collects two metrics: *pg_probackup.dir.size[#backup_directory]* (the size of the target directory) and *pg_probackup.dir.error[#backup_directory]* (backup errors) for each specified *backup_directory*.
223+
224+
If any generated backup has bad status, like ERROR, CORRUPT, ORPHAN, а trigger is fired.

0 commit comments

Comments
 (0)