Skip to content

Ensure that Puppet runs are idempotent #987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 1, 2018
Merged

Ensure that Puppet runs are idempotent #987

merged 1 commit into from
Mar 1, 2018

Conversation

joshuaspence
Copy link
Contributor

Currently this module requires two executions of puppet apply to properly configure MySQL. On an Ubuntu 16.04 instance, I am seeing the second Puppet run produce the following changes:

Notice: Compiled catalog for ubuntu-16.04-x64 in environment production in 1.93 seconds
Info: Applying configuration version '1504824440'
Info: Computing checksum on file /etc/mysql/conf.d/mysql.cnf
Info: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysql.cnf]: Filebucketed /etc/mysql/conf.d/mysql.cnf to puppet with sum c992f7b57d778f607e8d83792936e92b
Notice: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysql.cnf]/ensure: removed
Notice: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]/ensure: defined content as '{md5}59e52aeed142f9004c9e74ff5e4f1d84'
Info: Class[Mysql::Server::Config]: Scheduling refresh of Class[Mysql::Server::Service]
Info: Class[Mysql::Server::Service]: Scheduling refresh of Service[mysqld]
Info: Class[Mysql::Server::Service]: Scheduling refresh of Exec[wait_for_mysql_socket_to_open]
Notice: /Stage[main]/Mysql::Server::Service/Service[mysqld]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Mysql::Server::Service/Exec[wait_for_mysql_socket_to_open]: Triggered 'refresh' from 1 events
Notice: Applied catalog in 2.49 seconds

The problem is that Class['mysql::server::config'] is included before Class['mysql::server::install']. This means that /etc/mysql/my.cnf is configured first, but then subsequently overridden by the default configuration that is installed by the mysql-server package. This isn't picked up by the existing acceptance tests because they explicitly set config_file to '#{@tmpdir}/my.cnf.

@joshuaspence
Copy link
Contributor Author

I tried adding acceptance tests for this, but it's really tricky to get right because of the fact that the tests all run on the same host.

@joshuaspence
Copy link
Contributor Author

Actually, it can easily be reproduced with BEAKER_set=default bundle exec rspec spec/acceptance/mysql_server_spec.rb:4.

@joshuaspence
Copy link
Contributor Author

> BEAKER_set=default bundle exec rspec spec/acceptance/mysql_server_spec.rb:4
/usr/local/rvm/gems/ruby-2.3.4/gems/beaker-rspec-6.1.0/lib/beaker-rspec/helpers/serverspec.rb:43: warning: already initialized constant Module::VALID_OPTIONS_KEYS
/usr/local/rvm/gems/ruby-2.3.4/gems/specinfra-2.67.3/lib/specinfra/configuration.rb:4: warning: previous definition of VALID_OPTIONS_KEYS was here
Beaker::Hypervisor, found some vagrant boxes to create
Bringing machine 'ubuntu-1404-x64' up with 'virtualbox' provider...
==> ubuntu-1404-x64: Importing base box 'puppetlabs/ubuntu-14.04-64-nocm'...
Progress: 10%Progress: 20%Progress: 30%Progress: 40%Progress: 50%Progress: 60%Progress: 70%Progress: 80%Progress: 90%==> ubuntu-1404-x64: Matching MAC address for NAT networking...
==> ubuntu-1404-x64: Checking if box 'puppetlabs/ubuntu-14.04-64-nocm' is up to date...
==> ubuntu-1404-x64: Setting the name of the VM: defaultyml_ubuntu-1404-x64_1505218683813_91418
==> ubuntu-1404-x64: Clearing any previously set network interfaces...
==> ubuntu-1404-x64: Preparing network interfaces based on configuration...
    ubuntu-1404-x64: Adapter 1: nat
    ubuntu-1404-x64: Adapter 2: hostonly
==> ubuntu-1404-x64: Forwarding ports...
    ubuntu-1404-x64: 22 (guest) => 2222 (host) (adapter 1)
==> ubuntu-1404-x64: Running 'pre-boot' VM customizations...
==> ubuntu-1404-x64: Booting VM...
==> ubuntu-1404-x64: Waiting for machine to boot. This may take a few minutes...
    ubuntu-1404-x64: SSH address: 127.0.0.1:2222
    ubuntu-1404-x64: SSH username: vagrant
    ubuntu-1404-x64: SSH auth method: private key
==> ubuntu-1404-x64: Machine booted and ready!
==> ubuntu-1404-x64: Checking for guest additions in VM...
    ubuntu-1404-x64: The guest additions on this VM do not match the installed version of
    ubuntu-1404-x64: VirtualBox! In most cases this is fine, but in rare cases it can
    ubuntu-1404-x64: prevent things such as shared folders from working properly. If you see
    ubuntu-1404-x64: shared folder errors, please make sure the guest additions within the
    ubuntu-1404-x64: virtual machine match the version of VirtualBox you have installed on
    ubuntu-1404-x64: your host and reload your VM.
    ubuntu-1404-x64: 
    ubuntu-1404-x64: Guest Additions Version: 5.0.16
    ubuntu-1404-x64: VirtualBox Version: 5.1
==> ubuntu-1404-x64: Setting hostname...
==> ubuntu-1404-x64: Configuring and enabling network interfaces...
==> ubuntu-1404-x64: Mounting shared folders...
    ubuntu-1404-x64: /vagrant => /home/josh/workspace/puppetlabs-mysql/.vagrant/beaker_vagrant_files/default.yml
Disabling updates.puppetlabs.com by modifying hosts file to resolve updates to 127.0.0.1 on ubuntu-1404-x64
localhost $ scp /home/josh/workspace/puppetlabs-mysql ubuntu-1404-x64:/etc/puppetlabs/code/modules {:ignore => [".bundle", ".git", ".idea", ".vagrant", ".vendor", "vendor", "acceptance", "bundle", "spec", "tests", "log", ".svn", "junit", "pkg", "example", ".", ".."]}
Run options: include {:locations=>{"./spec/acceptance/mysql_server_spec.rb"=>[4]}}

mysql class
  advanced config
    behaves like a idempotent resource
localhost $ scp /tmp/beaker20170912-28605-4xr9rh ubuntu-1404-x64:/tmp/apply_manifest.pp.Qo7IMu {:ignore => }
      should apply with no errors
localhost $ scp /tmp/beaker20170912-28605-1q7wvn ubuntu-1404-x64:/tmp/apply_manifest.pp.RZ5mbZ {:ignore => }
      should apply a second time without changes (FAILED - 1)
Destroying vagrant boxes
==> ubuntu-1404-x64: Forcing shutdown of VM...
==> ubuntu-1404-x64: Destroying VM and associated drives...

Failures:

  1) mysql class advanced config behaves like a idempotent resource should apply a second time without changes
     Failure/Error: apply_manifest(pp, :catch_changes => true)
     Beaker::Host::CommandFailure:
       Host 'ubuntu-1404-x64' exited with 2 running:
        puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.RZ5mbZ
       Last 10 lines of output were:
       	Notice: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/.keepme]/ensure: removed
       	Info: Computing checksum on file /etc/mysql/conf.d/mysqld_safe_syslog.cnf
       	Info: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysqld_safe_syslog.cnf]: Filebucketed /etc/mysql/conf.d/mysqld_safe_syslog.cnf to puppet with sum 347fa9f3d579ffc06c39bf7e69f4fc6e
       	Notice: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysqld_safe_syslog.cnf]/ensure: removed
       	Info: Class[Mysql::Server::Config]: Scheduling refresh of Class[Mysql::Server::Service]
       	Info: Class[Mysql::Server::Service]: Scheduling refresh of Service[mysqld]
       	Info: Class[Mysql::Server::Service]: Scheduling refresh of Exec[wait_for_mysql_socket_to_open]
       	Notice: /Stage[main]/Mysql::Server::Service/Service[mysqld]: Triggered 'refresh' from 1 events
       	Notice: /Stage[main]/Mysql::Server::Service/Exec[wait_for_mysql_socket_to_open]: Triggered 'refresh' from 1 events
       	Notice: Applied catalog in 3.51 seconds
       
     Shared Example Group: "a idempotent resource" called from ./spec/acceptance/mysql_server_spec.rb:51
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/host.rb:375:in `exec'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-puppet-0.6.0/lib/beaker-puppet/helpers/puppet_helpers.rb:505:in `block in apply_manifest_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-3.24.0/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-puppet-0.6.0/lib/beaker-puppet/helpers/puppet_helpers.rb:434:in `apply_manifest_on'
     # /usr/local/rvm/gems/ruby-2.3.4/gems/beaker-puppet-0.6.0/lib/beaker-puppet/helpers/puppet_helpers.rb:512:in `apply_manifest'
     # ./spec/spec_helper_acceptance.rb:45:in `block (2 levels) in <top (required)>'

Finished in 2 minutes 13.4 seconds (files took 4 minutes 4.6 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/acceptance/mysql_server_spec.rb[1:1:1:2] # mysql class advanced config behaves like a idempotent resource should apply a second time without changes

Currently this module requires two executions of `puppet apply` to properly configure MySQL. On an Ubuntu 16.04 instance, I am seeing the second Puppet run produce the following changes:

```
Notice: Compiled catalog for ubuntu-16.04-x64 in environment production in 1.93 seconds
Info: Applying configuration version '1504824440'
Info: Computing checksum on file /etc/mysql/conf.d/mysql.cnf
Info: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysql.cnf]: Filebucketed /etc/mysql/conf.d/mysql.cnf to puppet with sum c992f7b57d778f607e8d83792936e92b
Notice: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/conf.d/mysql.cnf]/ensure: removed
Notice: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]/ensure: defined content as '{md5}59e52aeed142f9004c9e74ff5e4f1d84'
Info: Class[Mysql::Server::Config]: Scheduling refresh of Class[Mysql::Server::Service]
Info: Class[Mysql::Server::Service]: Scheduling refresh of Service[mysqld]
Info: Class[Mysql::Server::Service]: Scheduling refresh of Exec[wait_for_mysql_socket_to_open]
Notice: /Stage[main]/Mysql::Server::Service/Service[mysqld]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Mysql::Server::Service/Exec[wait_for_mysql_socket_to_open]: Triggered 'refresh' from 1 events
Notice: Applied catalog in 2.49 seconds
```

The problem is that `Class['mysql::server::config']` is included before `Class['mysql::server::install']`. This means that `/etc/mysql/my.cnf` is configured first, but then subsequently overridden by the default configuration that is installed by the `mysql-server` package. This isn't picked up by the existing acceptance tests because they explicitly set `config_file` to `'#{@tmpdir}/my.cnf`.
@david22swan david22swan merged commit 7a77084 into puppetlabs:master Mar 1, 2018
@joshuaspence joshuaspence deleted the ordering branch March 3, 2018 08:17
@baurmatt
Copy link
Contributor

baurmatt commented Apr 3, 2019

Same for us with Percona Server on Ubuntu 1[468].04. See https://tickets.puppetlabs.com/browse/MODULES-8701.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants