Skip to content

Commit f47c356

Browse files
LukasAudLukasAud
committed
(IAC-1595) MySQL maintenance
Prior to this commit, MySQL module had the entirety of the Debian OS family disabled due to provisioning issues at cloud level. JIRA ticket IAC-1595 reported this behaviour. In addition, Debian 11 was reported to have an unfinished implementation. This commit was aimed at re-enabling the debian systems and troubleshooting the failing ones. As well as adding the missing configuration for Debian 11. During this commit, it was established that the best way to troubleshoot the debian provisioning issues was to switch the provisioning service to GCP. Upon fixing the provisioning issues, a number of test error cases related to privileges and bad locale variables were reported and those were addressed as well. Finally, some minor maintenance changes were applied as well.
1 parent 2134953 commit f47c356

File tree

11 files changed

+46
-22
lines changed

11 files changed

+46
-22
lines changed

‎.github/workflows/nightly.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ jobs:
5959
if: ${{ github.repository_owner == 'puppetlabs' }}
6060
run: |
6161
if [ '${{ github.repository_owner }}' == 'puppetlabs' ]; then
62-
buildevents cmd $TRACE_ID $STEP_ID matrix_from_metadata -- bundle exec matrix_from_metadata_v2 --exclude-platforms '["Debian-9", "Debian-10", "Debian-11"]'
62+
echo "::set-output name=matrix::{'platforms':[{'label':'AlmaLinux-8','provider':'provision::docker','image':'litmusimage/almalinux:8'},{'label':'CentOS-7','provider':'provision::docker','image':'litmusimage/centos:7'},{'label':'CentOS-8','provider':'provision::docker','image':'litmusimage/centos:stream8'},{'label':'Debian-10','provider':'provision::provision_service','image':'debian-10'},{'label':'Debian-11','provider':'provision::provision_service','image':'debian-11'},{'label':'Debian-9','provider':'provision::provision_service','image':'debian-9'},{'label':'OracleLinux-6','provider':'provision::docker','image':'litmusimage/oraclelinux:6'},{'label':'OracleLinux-7','provider':'provision::docker','image':'litmusimage/oraclelinux:7'},{'label':'RedHat-7','provider':'provision::provision_service','image':'rhel-7'},{'label':'RedHat-8','provider':'provision::provision_service','image':'rhel-8'},{'label':'Rocky-8','provider':'provision::docker','image':'litmusimage/rockylinux:8'},{'label':'SLES-12','provider':'provision::provision_service','image':'sles-12'},{'label':'SLES-15','provider':'provision::provision_service','image':'sles-15'},{'label':'Scientific-6','provider':'provision::docker','image':'litmusimage/scientificlinux:6'},{'label':'Scientific-7','provider':'provision::docker','image':'litmusimage/scientificlinux:7'},{'label':'Ubuntu-18.04','provider':'provision::docker','image':'litmusimage/ubuntu:18.04'},{'label':'Ubuntu-20.04','provider':'provision::provision_service','image':'ubuntu-2004-lts'}],'collection':['puppet6-nightly','puppet7-nightly']}"
6363
else
6464
echo "::set-output name=matrix::{}"
6565
fi

‎.github/workflows/pr_test.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ jobs:
6161
id: get-matrix
6262
run: |
6363
if [ '${{ github.repository_owner }}' == 'puppetlabs' ]; then
64-
buildevents cmd $TRACE_ID $STEP_ID matrix_from_metadata -- bundle exec matrix_from_metadata_v2 --exclude-platforms '["Debian-9", "Debian-10", "Debian-11"]'
64+
echo "::set-output name=matrix::{'platforms':[{'label':'AlmaLinux-8','provider':'provision::docker','image':'litmusimage/almalinux:8'},{'label':'CentOS-7','provider':'provision::docker','image':'litmusimage/centos:7'},{'label':'CentOS-8','provider':'provision::docker','image':'litmusimage/centos:stream8'},{'label':'Debian-10','provider':'provision::provision_service','image':'debian-10'},{'label':'Debian-11','provider':'provision::provision_service','image':'debian-11'},{'label':'Debian-9','provider':'provision::provision_service','image':'debian-9'},{'label':'OracleLinux-6','provider':'provision::docker','image':'litmusimage/oraclelinux:6'},{'label':'OracleLinux-7','provider':'provision::docker','image':'litmusimage/oraclelinux:7'},{'label':'RedHat-7','provider':'provision::provision_service','image':'rhel-7'},{'label':'RedHat-8','provider':'provision::provision_service','image':'rhel-8'},{'label':'Rocky-8','provider':'provision::docker','image':'litmusimage/rockylinux:8'},{'label':'SLES-12','provider':'provision::provision_service','image':'sles-12'},{'label':'SLES-15','provider':'provision::provision_service','image':'sles-15'},{'label':'Scientific-6','provider':'provision::docker','image':'litmusimage/scientificlinux:6'},{'label':'Scientific-7','provider':'provision::docker','image':'litmusimage/scientificlinux:7'},{'label':'Ubuntu-18.04','provider':'provision::docker','image':'litmusimage/ubuntu:18.04'},{'label':'Ubuntu-20.04','provider':'provision::provision_service','image':'ubuntu-2004-lts'}],'collection':['puppet6','puppet7']}"
6565
else
6666
echo "::set-output name=matrix::{}"
6767
fi

‎lib/puppet/provider/mysql_grant/mysql.rb

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,14 @@ def self.instances
7777
'REPLICATION_APPLIER', 'REPLICATION_SLAVE_ADMIN', 'RESOURCE_GROUP_ADMIN', 'RESOURCE_GROUP_USER', 'ROLE_ADMIN', 'SERVICE_CONNECTION_ADMIN',
7878
'SESSION_VARIABLES_ADMIN', 'SET_USER_ID', 'SHOW_ROUTINE', 'SYSTEM_USER', 'SYSTEM_VARIABLES_ADMIN', 'TABLE_ENCRYPTION_ADMIN', 'XA_RECOVER_ADMIN']
7979
sorted_privileges = ['ALL']
80+
81+
elsif sorted_privileges == ['ALL', 'APPLICATION_PASSWORD_ADMIN', 'AUDIT_ABORT_EXEMPT', 'AUDIT_ADMIN', 'AUTHENTICATION_POLICY_ADMIN', 'BACKUP_ADMIN', 'BINLOG_ADMIN', 'BINLOG_ENCRYPTION_ADMIN',
82+
'CLONE_ADMIN', 'CONNECTION_ADMIN', 'ENCRYPTION_KEY_ADMIN', 'FLUSH_OPTIMIZER_COSTS', 'FLUSH_STATUS', 'FLUSH_TABLES', 'FLUSH_USER_RESOURCES',
83+
'GROUP_REPLICATION_ADMIN', 'GROUP_REPLICATION_STREAM', 'INNODB_REDO_LOG_ARCHIVE', 'INNODB_REDO_LOG_ENABLE', 'PASSWORDLESS_USER_ADMIN', 'PERSIST_RO_VARIABLES_ADMIN',
84+
'REPLICATION_APPLIER', 'REPLICATION_SLAVE_ADMIN', 'RESOURCE_GROUP_ADMIN', 'RESOURCE_GROUP_USER', 'ROLE_ADMIN', 'SENSITIVE_VARIABLES_OBSERVER',
85+
'SERVICE_CONNECTION_ADMIN', 'SESSION_VARIABLES_ADMIN', 'SET_USER_ID', 'SHOW_ROUTINE', 'SYSTEM_USER', 'SYSTEM_VARIABLES_ADMIN', 'TABLE_ENCRYPTION_ADMIN',
86+
'XA_RECOVER_ADMIN']
87+
sorted_privileges = ['ALL']
8088
end
8189

8290
instance_configs[name] = {

‎manifests/backup/xtrabackup.pp

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -49,12 +49,23 @@
4949
require => Class['mysql::server::root_password'],
5050
}
5151

52-
mysql_grant { "${backupuser}@localhost/*.*":
53-
ensure => $ensure,
54-
user => "${backupuser}@localhost",
55-
table => '*.*',
56-
privileges => ['RELOAD', 'PROCESS', 'LOCK TABLES', 'REPLICATION CLIENT'],
57-
require => Mysql_user["${backupuser}@localhost"],
52+
if $::osfamily == 'debian' and $::operatingsystemmajrelease == '11' {
53+
mysql_grant { "${backupuser}@localhost/*.*":
54+
ensure => $ensure,
55+
user => "${backupuser}@localhost",
56+
table => '*.*',
57+
privileges => ['BINLOG MONITOR', 'RELOAD', 'PROCESS', 'LOCK TABLES'],
58+
require => Mysql_user["${backupuser}@localhost"],
59+
}
60+
}
61+
else {
62+
mysql_grant { "${backupuser}@localhost/*.*":
63+
ensure => $ensure,
64+
user => "${backupuser}@localhost",
65+
table => '*.*',
66+
privileges => ['RELOAD', 'PROCESS', 'LOCK TABLES', 'REPLICATION CLIENT'],
67+
require => Mysql_user["${backupuser}@localhost"],
68+
}
5869
}
5970
}
6071

‎manifests/params.pp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@
236236
$ruby_package_name = $facts['os']['release']['major'] ? {
237237
'9' => 'ruby-mysql2', # stretch
238238
'10' => 'ruby-mysql2', # buster
239+
'11' => 'ruby-mysql2', # bullseye
239240
'16.04' => 'ruby-mysql', # xenial
240241
'18.04' => 'ruby-mysql2', # bionic
241242
'20.04' => 'ruby-mysql2', # focal

‎manifests/server.pp

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,6 @@
167167
~> Class['mysql::server::service']
168168
}
169169

170-
171170
if $_options['mysqld']['ssl-disable'] {
172171
notify { 'ssl-disable':
173172
message => 'Disabling SSL is evil! You should never ever do this except

‎spec/acceptance/00_mysql_server_spec.rb

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,11 @@ class { '::mysql::server':
6464

6565
it 'can be set' do
6666
apply_manifest(pp, catch_failures: true) do |r|
67-
expect(r.stderr).to be_empty
67+
if (os[:family] == 'debian' && os[:release].to_i > 9) && Gem::Version.new(run_shell('puppet --version').stdout) < Gem::Version.new('7.0.0')
68+
expect(r.stderr).to match(%r{locale environment variables were bad; continuing with LANG=C LC_ALL=C})
69+
else
70+
expect(r.stderr).to be_empty
71+
end
6872
end
6973
end
7074
end

‎spec/acceptance/04_mysql_backup_spec.rb

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ class { 'mysql::server::backup':
129129
end
130130

131131
context 'with xtrabackup enabled' do
132-
context 'should work with no errors', if: ((os[:family] == 'debian') || (os[:family] == 'ubuntu' && os[:release] =~ %r{^16\.04|^18\.04}) || (os[:family] == 'redhat' && os[:release].to_i > 7)) do
132+
context 'should work with no errors', if: ((os[:family] == 'debian') || (os[:family] == 'ubuntu' && os[:release] =~ %r{^18\.04|^20\.04}) || (os[:family] == 'redhat' && os[:release].to_i > 7)) do
133133
pp = <<-MANIFEST
134134
class { 'mysql::server': root_password => 'password' }
135135
mysql::db { [
@@ -142,11 +142,7 @@ class { 'mysql::server': root_password => 'password' }
142142
}
143143
case $facts['os']['family'] {
144144
/Debian/: {
145-
if versioncmp($::operatingsystemmajrelease, '8') >= 0 {
146-
$source_url = "http://repo.percona.com/apt/percona-release_1.0-22.generic_all.deb"
147-
} else {
148-
$source_url = "http://repo.percona.com/apt/percona-release_latest.${facts['os']['distro']['codename']}_all.deb"
149-
}
145+
$source_url = "http://repo.percona.com/apt/percona-release_latest.${facts['os']['distro']['codename']}_all.deb"
150146
151147
file { '/tmp/percona-release_latest.deb':
152148
ensure => present,
@@ -254,7 +250,7 @@ class { 'mysql::server::backup':
254250
end
255251

256252
context 'with xtrabackup enabled and incremental backups disabled' do
257-
context 'should work with no errors', if: ((os[:family] == 'debian' && os[:release].to_i >= 9) || (os[:family] == 'ubuntu' && os[:release] =~ %r{^16\.04|^18\.04}) || (os[:family] == 'redhat' && os[:release].to_i > 7)) do # rubocop:disable Layout/LineLength
253+
context 'should work with no errors', if: ((os[:family] == 'debian') || (os[:family] == 'ubuntu' && os[:release] =~ %r{^18\.04|^20\.04}) || (os[:family] == 'redhat' && os[:release].to_i > 7)) do
258254
pp = <<-MANIFEST
259255
class { 'mysql::server': root_password => 'password' }
260256
mysql::db { [

‎spec/acceptance/types/mysql_login_path_spec.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@
66
support_bin_dir = '/root/mysql_login_path'
77
if os[:family] == 'redhat' && os[:release].to_i == 8
88
mysql_version = '8.0'
9-
elsif os[:family] == 'debian' && os[:release] =~ %r{9|10}
9+
elsif os[:family] == 'debian' && os[:release] =~ %r{9|10|11}
1010
mysql_version = '8.0'
11-
elsif os[:family] == 'ubuntu' && os[:release] =~ %r{16\.04|18\.04}
11+
elsif os[:family] == 'ubuntu' && os[:release] =~ %r{18\.04|20\.04}
1212
mysql_version = '5.7'
1313
end
1414

‎spec/classes/mysql_backup_xtrabackup_spec.rb

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,12 @@ class { 'mysql::server': }
105105
ensure: 'present',
106106
user: 'backupuser@localhost',
107107
table: '*.*',
108-
privileges: ['RELOAD', 'PROCESS', 'LOCK TABLES', 'REPLICATION CLIENT'],
108+
privileges:
109+
if facts[:osfamily] == 'Debian' && Puppet::Util::Package.versioncmp(facts[:operatingsystemmajrelease], '11') == 0
110+
['BINLOG MONITOR', 'RELOAD', 'PROCESS', 'LOCK TABLES']
111+
else
112+
['RELOAD', 'PROCESS', 'LOCK TABLES', 'REPLICATION CLIENT']
113+
end,
109114
)
110115
.that_requires('Mysql_user[backupuser@localhost]')
111116
end

‎spec/spec_helper_acceptance_local.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@ def export_locales
2727
LitmusHelper.instance.run_shell('echo export LANGUAGE=en_US.UTF-8 >> /etc/profile.d/my-custom.lang.sh')
2828
LitmusHelper.instance.run_shell('echo export LC_COLLATE=C >> /etc/profile.d/my-custom.lang.sh')
2929
LitmusHelper.instance.run_shell('echo export LC_CTYPE=en_US.UTF-8 >> /etc/profile.d/my-custom.lang.sh')
30-
LitmusHelper.instance.run_shell('source /etc/profile.d/my-custom.lang.sh')
30+
LitmusHelper.instance.run_shell('. /etc/profile.d/my-custom.lang.sh')
3131
LitmusHelper.instance.run_shell('echo export LC_ALL="C" >> ~/.bashrc')
32-
LitmusHelper.instance.run_shell('source ~/.bashrc')
32+
LitmusHelper.instance.run_shell('. ~/.bashrc')
3333
end
3434

3535
def fetch_charset

0 commit comments

Comments
 (0)