Build an Ohai plugin

In this tutorial, you'll learn how to create, test, and deploy an Ohai plugin that will be able to execute a system command and collect data about an installed application on a node.

Ohai collects data about your system. This data can be used to make your recipes more dynamic or provide better reporting. Each piece of data is captured through various plugins that are defined and shipped with Ohai. These plugins are defined with a domain-specific language (DSL) that is written in Ruby (similar to defining a recipe or a custom resource).

You can create your own Ohai plugins. These plugins can collect important data about your specialized hardware, virtualization, environment, and software. This data is collected at the start of the chef-client run and then stored as automatic attributes on the node. These node attributes can be used within your recipes or used in reporting when the node object is sent off to the Chef Server.

  This example is specific to Apache running on CentOS 6.7 but nearly all the practices are easily extendable to other applications and platforms.

Here's a 30 minute video that explains how to build an Ohai plugin in detail that you can watch before or after you complete this tutorial.


Prerequisites

Understand hardware requirements

To successfully complete this tutorial, you'll need a Windows, MacOS, or Linux workstation that has relatively powerful hardware. We recommend that your workstation have at least 15GB of free disk space and at least 4GB of available memory so that you can download the base image and run a few virtual machine instances.

Set up your workstation for local development

This tutorial builds on top off knowledge gained in Get started with Test Kitchen and Develop a web app cookbook.

First, you'll need to have the latest version of the Chef Development Kit on your workstation. You'll also need a programmer's text editor that you're comfortable working with and a directory to work from – the tutorials use ~/learn-chef. Follow these steps to get set up.

You'll also need the virtualization software VirtualBox and Vagrant. If you don't have VirtualBox and Vagrant installed, follow steps 1—3 to get set up. Optionally, you can follow steps 4—7 to verify you can successfully bring up a local virtual machine.

Build local development skills using Test Kitchen

This tutorial requires familiarity with local development using Test Kitchen.

If you're not familiar with Test Kitchen, work through Get started with Test Kitchen before you start this tutorial. This tutorial teaches you the basics of how to apply cookbooks on local test instances, all on your workstation.

The Develop a web app cookbook tutorial provides additional hands-on practice with local development. Working through this tutorial is optional. Here, you configure a basic but complete web application using Test Kitchen. The quickstart brings up the final configuration on a local test instance in just a few minutes.

1. Creating the Ohai plugin

An Ohai plugin is a Ruby file that needs to exist on the node. We will create a plugin file within our application cookbook and talk more about delivering the plugin to our system in the 'Deploying the plugin' section.

Create an application cookbook

From your terminal window, begin by moving to the ~/learn-chef directory.

Terminal: ~

$
cd ~/learn_chef

Next, run the following chef generate cookbook cookbooks/apache command to create a cookbook named apache under the ~/learn-chef/cookbooks directory.

Terminal: ~/learn-chef

$                 
chef generate cookbook cookbooks/apacheGenerating cookbook apache- Ensuring correct cookbook file content- Ensuring delivery configuration- Ensuring correct delivery build cookbook content Your cookbook is ready. Type \`cd cookbooks/apache\` to enter it. There are several commands you can run to get started locally developing and testing your cookbook.Type \`delivery local --help\` to see a full list. Why not start by writing a test? Tests for the default recipe are stored at: test/recipes/default_test.rb If you\'d prefer to dive right in, the default recipe can be found at: recipes/default.rb

Next, cd to your cookbook directory.

Terminal: ~/learn-chef

$
cd ~/learn-chef/cookbooks/apache

Install the application and start the service in the default recipe.

Editor: ~/learn-chef/cookbooks/apache/recipes/default.rb

1
2
3
4
5
package 'httpd'

service 'httpd' do
  action [:start, :enable]
end

Create the Ohai plugin

Run chef generate file apache_modules.rb to create a cookbook file named apache_modules under the ~/learn-chef/cookbooks/apache/files/default directory.

Terminal: ~/learn-chef/cookbooks/apache

$
chef generate file apache_modules.rb

Write out the plugin using Ohai's DSL:

Editor: ~/learn-chef/cookbooks/apache/files/default/apache_modules.rb

1
2
3
4
5
6
7
8
9
Ohai.plugin :Apache do
  provides 'apache/modules'

  collect_data :default do
    apache(Mash.new)
    modules_cmd = shell_out('apachectl -t -D DUMP_MODULES')
    apache[:modules] = modules_cmd.stdout
  end
end

Let's talk about the syntax of an Ohai plugin:

  • An Ohai plugin name is always a symbol that starts with a capital letter, for example, :Apache.
  • provides specifies where the content can be found on the node object, for example, node['apache']['modules'].
  • This collect_data block is the default way we want to collect the information; you can also define platform-specific collect_data blocks.
  • A Mash is a Hash that allows you to use string keys and symbol keys interchangebly to retreive the same value.
  • shell_out is a helper method that executes a command; returning an object that we can query the standard out (stdout), standard error (stderr), and exit status code (exitstatus).

2. Testing the Ohai plugin with ChefSpec

To ensure our plugin works would require us to deliver it to our target node and then execute ohai or chef-client.

When an Ohai plugin fails it does so silently so that it does not effect the remainder of the ohai execution and the chef-client run. If you were to make a mistake while creating or changing the plugin it would take a long time to troubleshoot the issue.

  On success, we would be able to see the data that our collect_data block has captured about our node.
  On failure, no data will be captured and no output will tell us what went wrong.

Before we deploy this plugin we will test it locally. This can be done first through ChefSpec with the help of the chefspec-ohai gem.

Install and require the chefspec-ohai gem

An additional gem is needed if we want to test our Ohai plugin through ChefSpec.

Terminal: ~/learn-chef/cookbooks/apache

$   
chef gem install chefspec-ohaiFetching: chefspec-ohai-0.1.0.gem (100%)Successfully installed chefspec-ohai-0.1.01 gem installed

With the gem installed, now we need to load it within our test suite:

Editor: ~/learn-chef/cookbooks/apache/spec/spec_helper.rb

1
2
3
require 'chefspec'
require 'chefspec/berkshelf'
require 'chefspec/ohai'

Write a ohai plugin test

Create a new directory to store your Ohai plugin tests:

Terminal: ~/learn-chef/cookbooks/apache

$
mkdir -p spec/unit/ohai_plugins

Write out the following test:

Editor: ~/learn-chef/cookbooks/apache/spec/unit/ohai_plugins/apache_modules_spec.rb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
require 'spec_helper'

describe_ohai_plugin :Apache do
  let(:plugin_file) { 'files/default/apache_modules.rb' }

  it 'provides apache/modules' do
    expect(plugin).to provides_attribute('apache/modules')
  end

  let(:command) { double('Fake Command',stdout: 'OUTPUT') }

  it 'correctly captures output' do
    allow(plugin).to receive(:shell_out).with('apachectl -t -D DUMP_MODULES').and_return(command)
    expect(plugin_attribute('apache/modules')).to eq('OUTPUT')
  end
end

Let's talk about the syntax of an Ohai plugin:

  • describe_ohai_plugin is a new example group alias that allows you to easily test Ohai plugins.
  • let(:plugin_file) { ... } defines the relative path to the cookbook file that contains the Ohai plugin.
  • The first example asserts that the plugin provides the correct attribribute.
  • let(:command) { ... } returns the test double command object which we use in the next example.
  • double('Fake Command',stdout: 'OUTPUT') is an RSpec test double that takes a name and a hash of parameters. These parameters are converted to methods on the double object.
  • allow(plugin).to_receive(:shell_out).with('...') is RSpec's way of allowing messages to override the shell_out command when used within the plugin; replace the results of it when given the specified command.
  • plugin_attribute('apache/modules') retrieves the value stored in the modules key (similar to writing node['apache']['modules']).

Execute the ohai plugin test

Execute rspec with the new test to validate that the plugin is defined correctly:

Terminal: ~/learn-chef/cookbooks/apache

$     
chef exec rspec spec/ohai_plugins/apache_modules_spec.rb.[2016-12-02T14:18:59-06:00] WARN: Plugin Definition Error: <files/default>: collect_data already defined on platform default. Finished in 0.20469 seconds (files took 1.54 seconds to load)2 examples, 0 failures

3. Deploying the Ohai Plugin

We have the application cookbook with the plugin file defined as a cookbook file (apache_modules.rb). To deploy the plugin we need to:

  • copy the plugin to the correct file location.
  • ask Ohai to load the plugin we delivered.

The ohai community cookbook does both of these things extremely well through a custom resource named ohai_plugin.

Add the ohai community cookbook as a dependency and install dependencies

Append to the end of the cookbook's metadata the dependency on the ohai community cookbook.

Editor: ~/learn-chef/cookbooks/apache/metadata.rb

1
2
3
4
5
# ...
# If you upload to Supermarket you should set this so your cookbook
# gets a `View Source` link
# source_url 'https://github.com/<insert_org_here>/apache' if respond_to?(:source_url)
depends 'ohai'

Install the new dependencies:

Terminal: ~/learn-chef/cookbooks/apache

$     
berks installResolving cookbook dependencies...Fetching 'apache' from source at .Using apache (0.1.0) from source at .Using ohai (4.2.2)Using compat_resource (12.16.2)

Create and write a new recipe to deploy the plugin

Create a new recipe:

Terminal: ~/learn-chef/cookbooks/apache

$  
chef generate recipe ohai_apache_modulesRecipe: code_generator::recipe...

This Ohai plugin requires that the application be installed so we will include the default recipe first. Then we want to use the ohai_plugin resource and provide it the name of our Ohai plugin file.

Editor: ~/learn-chef/cookbooks/apache/recipes/ohai_apache_modules.rb

1
2
3
4
5
6
7
#
# Cookbook Name:: apache
# Recipe:: ohai_apache_modules
#
# Copyright (c) 2016 The Authors, All Rights Reserved.
include_recipe 'apache::default'
ohai_plugin 'apache_modules'

Add the recipe to your node's run list

This could be done directly to your node, including this recipe (and the cookbook as a dependency) within another recipe already on the run list, through a role or an environment.

4. Testing the Ohai plugin with InSpec

Testing our plugin with ChefSpec allowed us to ensure the plugin was defined correctly. It does not verify if the command we wrote works correctly on our given platform and platform version. This is where we can use Test Kitchen.

Add a new Test Kitchen suite that includes our plugin in the run list

To test this new recipe, create a new test suite within the Test Kitchen configuration.

Editor: ~/learn-chef/cookbooks/apache/.kitchen.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  always_update_cookbooks: true

verifier:
  name: inspec

platforms:
  - name: centos-6.7

suites:
  - name: default
    run_list:
      - recipe[apache::default]
    verifier:
      inspec_tests:
        - test/recipes
    attributes:
  # THIS IS THE NEW SUITE TO ADD:
  - name: ohai_apache_modules
    run_list:
      - recipe[apache::ohai_apache_modules]
    attributes:

Create and define a test that executes the plugin

The new suite we defined requires us to define a new location for our InSpec test.

Create the new test directory for the ohai_apache_modules test suite:

Terminal: ~/learn-chef/cookbooks/apache

$
mkdir -p test/recipes/ohai_apache_modules

Write the test:

Editor: test/recipes/ohai_apache_modules/default_test.rb

1
2
3
4
5
plugin_directory = '/tmp/kitchen/ohai/plugins'

describe command("ohai -d #{plugin_directory} apache") do
  its(:stdout) { should match(/core_module/) }
end
  • The ohai cookbook will store plugins in the specified directory within Test Kitchen.
  • The command resource is invoking the ohai command-line tool with the directory and specifying the plugin we want to view.
  • The standard out should displays all the modules; core_module is an example of one of those modules.

Execute this new test suite

Execute kitchen to create, converge and verify the new test suite.

Terminal: ~/learn-chef/cookbooks/apache

$
chef exec kitchen verify ohai_apache_modules

5. Refining Ohai plugin deployment

When the Test Kitchen converged our recipe a warning appeared:

Terminal: ~/learn-chef/cookbooks/apache

$            
chef exec kitchen converge......Compiling Cookbooks...Recipe: apache::ohai_httpd_modules  * ohai_plugin[httpd_modules] action create[2016-12-02T06:00:15+00:00] WARN: The Ohai  plugin_path does not include /tmp/kitchen/ohai/plugins. Ohai will reload on each  chef-client run in order to add this directory to the path unless you modify your  client.rb configuration to add this directory to plugin_path. See 'Ohai Settings' at https://docs.chef.io/config_rb_client.html   [2016-12-02T06:00:15+00:00] WARN: Adding /tmp/kitchen/ohai/plugins to the Ohai  plugin path for this chef-client run only  ...

The warning informs us that the ohai_plugin resource reloads Ohai during the chef-client run. This is inefficient. The warning gives a suggestion to modify the client.rb configuration file. This can be done with the chef-client community cookbook. This cookbook has a chef-client::config recipe which allows you to insert important configuration into the chef-client configuration file. That file is loaded before Ohai is executed during the chef-client run.

Outline of remaining work

  1. Add the chef-client cookbook to your cookbook repository.
  2. Add the chef-client::config recipe to the node's run-list.
  3. Set the node attribute node['ohai']['plugin_path'] to /etc/chef/ohai/plugins in a recipe, role, or environment.

Additional resources

Conclusion

Hopefully you see the power of creating an Ohai plugin to capture the important data about your node, have an understanding of why you should test the plugin locally with ChefSpec, and have an example cookbook that you can modify to suit your needs.