What is Kitchen and why would someone use it.

test-kitchen was originally written as a way to test chef cookbooks. But the provisioners and drivers are pluggable, kitchen-salt enables salt to be the provisioner instead of chef.

The goal of this kitchen-salt is to make it easy to test salt states or formulas independently of a production environment. It allows for doing quick checks of states and to make sure that upstream changes in packages will not affect deployments. By using platforms, users can run checks on their states against the environment they are running in production as well as checking future releases of distributions before doing major upgrades. It is also possible to test states against multiple versions of salt to make sure there are no major regressions.

Example formula

This article will be using my wordpress-formula to demo the major usage points of kitchen-salt.

Installing Kitchen

Most distributions provide a bundler gem in the repositories, but there are some that have a version of ruby that is too old to use kitchen. The easiest way to use kitchen on each system is to use a ruby version manager like rvm or rbenv. rbenv is very similar to pyenv.

Once ruby bundler is installed, it can be used to install localized versions of the ruby packages for each repository, using the bundle install command.

$ bundle install The latest bundler is 1.16.0.pre.2, but you are currently running 1.15.4. To update, run `gem install bundler --pre` Using artifactory 2.8.2 Using bundler 1.15.4 Using mixlib-shellout 2.3.2 Using mixlib-versioning 1.2.2 Using thor 0.19.1 Using net-ssh 4.2.0 Using safe_yaml 1.0.4 Using mixlib-install 2.1.12 Using net-scp 1.2.1 Using net-ssh-gateway 1.3.0 Using test-kitchen 1.17.0 Using kitchen-docker 2.6.1.pre from https://github.com/test-kitchen/kitchen-docker.git (at master@9eabd01) Using kitchen-salt 0.0.29 Bundle complete! 3 Gemfile dependencies, 13 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. 

This will require having a separate Gemfile to hold the requirements for running test-kitchen.

source "https://rubygems.org"  gem "test-kitchen" gem "kitchen-salt" gem 'kitchen-docker', :git => 'https://github.com/test-kitchen/kitchen-docker.git' 

Because I am also testing opensuse, right now the git version of kitchen-docker is required.

Using kitchen

$ bundle exec kitchen help Commands:   kitchen console                                 # Kitchen Console!   kitchen converge [INSTANCE|REGEXP|all]          # Change instance state to converge. Use a provisioner to configure one or more instances   kitchen create [INSTANCE|REGEXP|all]            # Change instance state to create. Start one or more instances   kitchen destroy [INSTANCE|REGEXP|all]           # Change instance state to destroy. Delete all information for one or more instances   kitchen diagnose [INSTANCE|REGEXP|all]          # Show computed diagnostic configuration   kitchen driver                                  # Driver subcommands   kitchen driver create [NAME]                    # Create a new Kitchen Driver gem project   kitchen driver discover                         # Discover Test Kitchen drivers published on RubyGems   kitchen driver help [COMMAND]                   # Describe subcommands or one specific subcommand   kitchen exec INSTANCE|REGEXP -c REMOTE_COMMAND  # Execute command on one or more instance   kitchen help [COMMAND]                          # Describe available commands or one specific command   kitchen init                                    # Adds some configuration to your cookbook so Kitchen can rock   kitchen list [INSTANCE|REGEXP|all]              # Lists one or more instances   kitchen login INSTANCE|REGEXP                   # Log in to one instance   kitchen package INSTANCE|REGEXP                 # package an instance   kitchen setup [INSTANCE|REGEXP|all]             # Change instance state to setup. Prepare to run automated tests. Install busser and related gems on one or more instances   kitchen test [INSTANCE|REGEXP|all]              # Test (destroy, create, converge, setup, verify and destroy) one or more instances   kitchen verify [INSTANCE|REGEXP|all]            # Change instance state to verify. Run automated tests on one or more instances   kitchen version                                 # Print Kitchen's version information 

The kitchen commands I use the most are: – list: show the current state of each configured environment – create: create the test environment with ssh or winrm. – converge: run the provision command, in this case, salt_solo and the specified states – verify: run the verifier. – login: login to created environment – destroy: remove the created environment – test: run create, converge, verify, and then destroy if it all succeeds

For triaging github issues, I regularly use bundle exec kitchen create <setup> and then salt bootstrap to install the salt version we are testing.

Then for running tests, to setup the environment I want to run the tests in I run bundle exec kitchen converge <setup>

Configuring test-kitchen

There are 6 major parts of the test-kitchen configuration file. This is .kitchen.yml and should be in the directory inside of which the kitchen command is going to be run.

  • driver: This specifies the configuration of how the driver requirements. Drivers are how the virtual machine is created. kitchen drivers (I prefer docker)
  • verifier: The command to run for tests to check that the converge ran successfully.
  • platforms: The different platforms/distributions to run on
  • transport: The transport layer to use to talk to the vm. This defaults to ssh, but winrm is also available.
  • suites: sets of different test runs.
  • provisioner: The plugin for provisioning the vm for the verifier to run against. This is where kitchen-salt comes in.

For the driver on the wordpress-fomula, the following is set:

driver:   name: docker   use_sudo: false   privileged: true   forward:     - 80 

This is using the kitchen-docker driver. If the user running kitchen does not have the correct privileges to run docker, then use_sudo: true should be set. All of the containers that are being used here are using systemd as the exec command, so privileged: true needs to be set. And then port 80 is forwarded to the host so that the verifier can run commands against it to check that wordpress has been setup

For the platforms, the following are setup to run systemd on the container start.

platforms:   - name: centos     driver_config:       run_command: /usr/lib/systemd/systemd   - name: opensuse     driver_config:       run_command: /usr/lib/systemd/systemd       provision_command:         - systemctl enable sshd.service   - name: ubuntu     driver_config:       run_command: /lib/systemd/systemd   - name: debian     driver_config:       run_command: /lib/systemd/systemd 

All of these distributions except for opensuse have sshd.service enabled when the package is installed, so we only have to have one provision command to enable sshd for opensuse. The rest have a command to configure the driver run_command to the correct systemd binary for that distribution.

For suites, there is only one suite.

suites:   - name: wordpress 

If multiple sets of pillars or different versions of salt were needed to be tested, they would be configured here.

suites:   - name: nitrogen   - name: develop     provisioner:       salt_bootstrap_options: -X -p git -p curl -p sudo git develop 

And there would be multiple suites with for each platform created and tested.

And lastly for the verifier.

verifier:   name: shell   remote_exec: false   command: pytest -v tests/integration/ 

There are a couple base verifiers. I usually use the shell verifier and use testinfra which has multiple connectors to run pytest type test functions inside of the container.

Kitchen also has a $KITCHEN_SUITE variable that it sets, so if different tests files need to be run for each suite.

verifier:   name: shell   remote_exec: false   command: pytest -v tests/integration/$KITCHEN_SUITE 

For the salt-jenkins, since we are setting up the containers to run the SaltStack testing suite, the verifier is setup to run inside of the container, and run the salt testing suite.

verifier:   name: shell   remote_exec: true   command: '$(kitchen) /testing/tests/runtests.py -v --output-columns=80 --run-destructive<%= ENV["TEST"] ? " -n #{ENV["TEST"]}" : "" %>' 

remote_exec will cause the command to be run inside of the container. The kitchen command uses the installed salt to lookup if py3 was used or not, so that the correct python executable is used to run the test suite. Then if the TEST environment variable is set, that test is run, otherwise the full test suite is run.

Configuring kitchen-salt

The documentation for kitchen-salt is located here

provisioner:   name: salt_solo   salt_install: bootstrap   salt_version: latest   salt_bootstrap_url: https://bootstrap.saltstack.com   salt_bootstrap_options: -X -p git -p curl -p sudo   is_file_root: true   require_chef: false   salt_copy_filter:     - .circleci/     - Dockerfile     - .drone.yml     - .git/     - .gitignore     - .kitchen/     - .kitchen.yml     - Gemfile     - Gemfile.lock     - requirements.txt     - tests/     - .travis.yml   dependencies:     - name: apache       repo: git       source: https://github.com/saltstack-formulas/apache-formula.git     - name: mysql       repo: git       source: https://github.com/saltstack-formulas/mysql-formula.git     - name: php       repo: git       source: https://github.com/saltstack-formulas/php-formula.git   state_top:     base:       "*":         - wordpress   pillars:     top.sls:       base:         "*":           - wordpress     wordpress.sls:       mysql:         database:           - wordpress         user:           wordpress:             password: quair9aiqueeShae4toh             host: localhost             databases:               - database: wordpress                 grants:                   - all privileges       wordpress:         lookup:           admin_user: gtmanfred           admin_email: daniel@gtmanfred.com           title: "GtManfred's Blog"           url: http://blog.manfred.io 
  • name: The name of the provisioner is salt_solo
  • salt_install: This defaults to bootstrap which installs using the salt bootstrap. Other options are apt and yum which use the repo.saltstack.com repository. ppa allows for specifying a ppa from which to install salt. And distrib which just uses whatever version of salt is provided by the distribution repositories.
  • salt_bootstrap_options: These are the bootstrap options that are passed to the bootstrap script. -X can be passed here to not start the salt services, because salt_solo runs salt-call and doesn't use the salt-minion process.
  • is_file_root: This is used to say just copy everything from the current directory to the tmp fileserver in the kitchen container. If there were not a custom module and state for this formula, kitchen could be set to have formula: wordpress to copy the wordpress-formula to the kitchen environment.
  • salt_copy_filter: This is a list of files to not copy to the kitchen environment.
  • dependencies: This is the fun part. If the formula depends on other formulas, they can be configured here. The following types are supported:
    • path – use a local path
    • git – clone a git repository
    • apt – install an apt package
    • yum – install a yum package
    • spm – install a spm package
  • state_top: This is the top file that will be used to run at the end of the provisioner
  • pillars: This is a set of custom pillars for configuring the instance. There are a couple other ways to provide pillars that are also useful.

Running test kitchen on pull requests.

Any of the major testing platforms should be usable. If there are complicated setups needed, Jenkins is probably the best, unfortunately I do not know jenkins very well, so I have provided examples for the three I know how to use.

  • TravisCI
  • Drone
  • CircleCI

My personal favorite is Drone. You can setup each one of the tests suites to run with a mysql container if you did not have states that need mysql-server installed on the instance. Also, for each job runner for Drone, you just need to setup another drone-agent on a server running docker, and then hook it into the drone-server, then each drone-agent can pick up a job and run it.

Original Article

This is the first edition of Arch monthly, mostly due to the lack of time to work on Arch weekly. So let’s start with the roundup of last month.

Two new Trusted Users

Alad and Foxboron joined the Trusted Users team! Congrats!

Archweb signoff helper

This has been around for a while, but Foxboron created this great tool to signoff packages in [testing] simply from the cli. If you are an official tester try it out!

Arch Classroom – Python for beginners

Pulec organizes a classroom about Python for beginners on Wednesday, October 04, 2017 at 16:00 UTC in the channel #archlinux-classroom on the freenode network. See this post for more details.

Eli Schwartz is our new bugwrangler

Eli Schwartz joins as bugwrangler by helping out assigning and investigation new bugs.

Arch-meson wrapper

If you package packages which uses meson as a build tool then the arch-meson is useful since it sets defaults for Arch.

Arch manpages website

A new website popped up which hosts Arch manual pages.

Arch monthly September was originally published by Jelle van der Waa at Jelly's Blog on October 02, 2017.

Original Article