Contributors1 hourIntroduction
In the module Getting Started With Policyfiles you saw how to create policyfiles and how they can be used to maintain run_lists and cookbook dependencies.
In this module we will see how we can use these in production in conjunction with a Chef Server to manage your environment.
This module picks up from where that previous module finishes, however you will download the material from that lab so its not necessary to complete it first.
Prerequisites
You'll need a Windows, macOS, or 64-bit Linux system with Chef Workstation installed.
You will need a separate Linux node that you can bootstrap with a policy file and manage with a Chef Server. This node could be a local VM (e.g. a Vagrant VM), or a VM running in a cloud (e.g. Azure or AWS). You will require the IP Address or FQDN of the VM as well as the root SSH credentials.
You will also need access to a Chef Server. If you don't have one readily available you can use the Hosted Chef Server Service. We will walk you through setting up a free account now.
1. Set Up Chef Server Account
For the purposes of this module we must have a Chef Server in place. Follow the steps in the Set Up Hosted Chef Account module to create a free account on Hosted Chef, and configure your workstation to communicate with it.
2. Clone the Repo
Navigate to a local working directory. On Linux or Mac you can use /tmp, as it is transient and all contents will be removed upon reboot, hence reducing clutter on your machine.
Run the following command to download the content we'll work with in this module
Terminal: ~
$$ | git clone https://github.com/learn-chef/lcr-policyfiles-managenodes.gitcd lcr-policyfiles-managenodes
|
If you have tree installed, you can run the following command to view the structure of the repo
Terminal: ~/lcr-policyfiles-managenodes
$ | tree -L 1 -a.├── .chef├── .chef-repo.txt├── .git├── .gitignore├── LICENSE├── README.md├── chefignore├── cookbooks├── data_bags└── policyfiles 5 directories, 5 files
|
You'll notice there is a policyfiles directory in your workspace as a sibling to cookbooks, and no roles and environments directories. When using policyfiles, roles and environments are obsolete.
You'll also notice a .chef directory in the cloned repo. This directory needs to contain the knife configuration file (knife.rb) and authorisation files (user.pem) required to communicate with the Chef Server. You should have downloaded these files in the previous step as part of setting up your Chef Server account.
Copy the files ~/learn-chef/.chef/knife.rb and ~/learn-chef/.chef/user.pem created in the previous step to this directory
| If you are using your own Chef sever along with the CLI, then the knife configuration file may be config.rb. Both knife.rb and config.rb are valid. |
Terminal: ~/lcr-policyfiles-managenodes
$ | cp ~/learn-chef/.chef/* ./.chef
|
3. Familiarize With the Base Cookbook
Let's take a couple minutes to familiarize with the base cookbook. This is the same cookbook you created if you completed the module Getting Started With Policyfiles.
The base cookbook in this repo is a wrapper for the hardening cookbook, and its purpose is to fix some security aspects of the OS. The hardening cookbook in turn has further dependencies on os-hardening and windows-hardening which are hosted here.
Open cookbooks/base/recipes/default.rb in your favorite editor.
Editor: cookbooks/base/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
| # Cookbook:: base
# Recipe:: default
#
# Copyright:: 2019, The Authors, All Rights Reserved.
include_recipe 'hardening::default'
file '/etc/motd' do
content node['base']['message']
end |
This recipe simply calls the hardening cookbook and updates the 'Message of the Day' file /etc/motd with a message set as an attribute.
This recipe gets invoked via the policyfile cookbooks/base/Policyfile.rb
Editor: cookbooks/base/Policyfile.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| # Policyfile.rb - Describe how you want Chef to build your system.
#
# For more information on the Policyfile feature, visit
# https://docs.chef.io/policyfile.html
# A name that describes what the system you're building with Chef does.
name 'base'
# Where to find external cookbooks:
default_source :supermarket
# run_list: chef-client will run these recipes in the order specified.
run_list 'base::default'
# Specify a custom source for a single cookbook:
cookbook 'base', path: '.'
cookbook 'hardening', path: '../hardening'
# Policyfile defined attributes
default['base']['message'] = "This node was hardened by Chef. Policyfile created at #{Time.now.utc}\n" |
4. Move and Rename the Policyfile
Policyfiles relate to the runlist and cookbook versions that get applied to a node. So, although our Policyfile.rb currently resides within our base cookbook, it may be more appropriate to maintain it separately in a different repo since it may refer to multiple nodes with different runlists, and the Policyfile.lock.json file may be updated more often than the cookbooks themselves.
Also, its using the default policyfile name of Policyfile.rb. In our case we are using a policyfile in a base OS level hardening cookbook, and we may decide later to nest this policyfile within other policyfiles, so lets rename it base-policy.rb so its more descriptive.
Run the following command to rename the policyfile as discussed and move it to the policyfiles directory.
Terminal: ~/lcr-policyfiles-managenodes
$ | mv cookbooks/base/Policyfile.rb policyfiles/base-policy.rb
|
Since you moved the policyfile, you will need to update it and change the path to the base and hardening cookbooks, as shown on lines 16 and 17 below
Editor: policyfiles/base-policy.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| # Policyfile.rb - Describe how you want Chef to build your system.
#
# For more information on the Policyfile feature, visit
# https://docs.chef.io/policyfile.html
# A name that describes what the system you're building with Chef does.
name 'base'
# Where to find external cookbooks:
default_source :supermarket
# run_list: chef-client will run these recipes in the order specified.
run_list 'base::default'
# Specify a custom source for a single cookbook:
cookbook 'base', path: '../cookbooks/base'
cookbook 'hardening', path: '../cookbooks/hardening'
# Policyfile defined attributes
default['base']['message'] = "This node was hardened by Chef. Policyfile created at #{Time.now.utc}\n" |
5. Install the base-policy.rb Policyfile
Run the chef install policyfile command to create the Policyfile.lock.json file.
Terminal: ~/lcr-policyfiles-managenodes
$ | chef install policyfiles/base-policy.rbBuilding policy baseExpanded run list: recipe[base::default]Caching Cookbooks...Installing base >= 0.0.0 from pathInstalling hardening >= 0.0.0 from pathUsing os-hardening 3.2.0Using windows-hardening 0.9.1Using windows-security-policy 0.3.9Using sysctl 1.0.5Using ohai 5.2.5 Lockfile written to /private/tmp/policy02/lcr-policyfiles-managenodes/policyfiles/base-policy.lock.jsonPolicy revision id: c0d724a6c86d8d7d1d5cfcf1de544c2d0476416016276c73b2bb3de2e4ef9690
|
6. Upload the Policyfile to Chef Server
When using policyfiles you do not use roles or environments - the policyfile pattern and the roles/environments patterns are incompatible.
Instead each node is placed in a 'policy group'. A policy group is a logical group of nodes that you want to have the same policy (similar to an environment). The policy group is assigned a policyfile that defines the run_list and all dependent cookbooks for the nodes in the policy group.
You should now have access to Hosted Chef, so let's first list the policyfiles on the Chef Server with the chef show-policy command
Terminal: ~/lcr-policyfiles-managenodes
$ | chef show-policyNo policies or policy groups exist on the server
|
The chef push command uploads an existing Policyfile.lock.json to a Chef Server, along with all the cookbooks referenced therein. The policy lock is applied to a specific POLICY_GROUP, which is a set of nodes that share the same run_list and cookbooks.
Terminal: ~/lcr-policyfiles-managenodes
$ | chef push acceptance policyfiles/base-policy.rbUploading policy base (c0d724a6c8) to policy group acceptanceUsing hardening 0.1.0 (87f4601f)Using ohai 5.2.5 (f393ae21)Using os-hardening 3.2.0 (1255dd1c)Using sysctl 1.0.5 (3d2a2314)Using windows-hardening 0.9.1 (fb514509)Using windows-security-policy 0.3.9 (b2e3ba1e)Uploaded base 0.1.0 (beeb9200)
|
Now run chef show-policy command again
Terminal: ~/lcr-policyfiles-managenodes
$ | chef show-policy base==== * acceptance: c0d724a6c8
|
This is showing the first ten characters of the revision_id of the base-policy.lock.json lock file, as illustrated below
Terminal: ~/lcr-policyfiles-managenodes
$ | grep c0d724a6c8 policyfiles/base-policy.lock.json "revision_id": "c0d724a6c86d8d7d1d5cfcf1de544c2d0476416016276c73b2bb3de2e4ef9690",
|
The revision_id is arguably the most important field in the lock file. It specifies the version of the policy you are running on the chef server and during your chef run. If you maintain your Policyfile.lock.json in a repository, e.g. GitHub, (as you should) then you'll always have a record of what runs on the node, and hence will always be able to 'roll back' to this version if necessary.
One important point to note when using policyfiles is with this pattern cookbooks are no longer required to be stored on the Chef Server. Instead the Policyfile.lock,json file defines the source location for chef-client to pull from. This may be on the Chef Server, but it could be direct from a source repository or from another artifact repository.
7. Bootstrap your node
Now that you have uploaded your policyfile to your Chef Server, its time to bootstrap a node.
Before we do so, you can verify the current nodes managed by Chef Server (if you just set up your Hosted Chef account as directed in this module, then there should be no existing nodes returned)
Terminal: ~/lcr-policyfiles-managenodes
Now run the knife bootstrap command below, replacing IPAddress, nodeusername and nodepassword with the relevant details of your node.
| The nodeusername should be a root user, or have sudo access. |
Terminal: ~/lcr-policyfiles-managenodes
$ | knife bootstrap IPAddress -x nodeusername -P nodepassword --sudo --policy-group acceptance --policy-name base -N mynode Creating new client for mynode Creating new node for mynode Connecting to 35.175.247.6 ... 35.175.247.6 Starting the first Chef Client run... 35.175.247.6 Starting Chef Client, version 14.9.13 35.175.247.6 Using policy 'base' at revision 'c0d724a6c86d8d7d1d5cfcf1de544c2d0476416016276c73b2bb3de2e4ef9690' 35.175.247.6 resolving cookbooks for run list: ["base::default@0.1.0 (78338ef)"] 35.175.247.6 Synchronizing Cookbooks: 35.175.247.6 - base (0.1.0) 35.175.247.6 - ohai (5.2.5) 35.175.247.6 - hardening (0.1.0) 35.175.247.6 - os-hardening (3.2.0) 35.175.247.6 - sysctl (1.0.5) 35.175.247.6 - windows-hardening (0.9.1) 35.175.247.6 - windows-security-policy (0.3.9) ... 35.175.247.6 Chef Client finished, 143/210 resources updated in 08 seconds
|
| You can disregard any deprecation warnings. |
You'll notice the Chef Client is using the same version of policy file we just uploaded, i.e. c0d724a6c86d8d7d1d5cfcf1de544c2d0476416016276c73b2bb3de2e4ef9690.
Now lets view details of the node on Chef Server
Terminal: ~/lcr-policyfiles-managenodes
$ | knife node show mynodeNode Name: mynodePolicy Name: basePolicy Group: acceptanceFQDN: ip-172-31-61-159.ec2.internalIP: 35.175.247.6Run List: recipe[base::default]Recipes: base::default, hardening::default, os-hardening::default, os-hardening::packages, os-hardening::yum, os-hardening::limits, os-hardening::login_defs, os-hardening::minimize_access, os-hardening::pam, os-hardening::profile, os-hardening::securetty, os-hardening::suid_sgid, os-hardening::sysctl, os-hardening::auditd, os-hardening::selinux, hardening::remediationPlatform: centos 7.6.1810Tags:
|
You'll see this node us using the base policy and is in the acceptance policy group.
Lets assume at this point our node has passed all relevant tests and we wish to move it to an environment called 'production'. First, we must assign our policyfile to the production policy group
Terminal: ~/lcr-policyfiles-managenodes
$ | chef push production policyfiles/base-policy.rbUploading policy base (f21e2334b6) to policy group productionUsing base 0.1.0 (beeb9200)Using hardening 0.1.0 (87f4601f)Using ohai 5.2.5 (f393ae21)Using os-hardening 3.2.0 (1255dd1c)Using sysctl 1.0.5 (3d2a2314)Using windows-hardening 0.9.1 (fb514509)Using windows-security-policy 0.3.9 (b2e3ba1e)
|
View the policies on Chef Server
Terminal: ~/lcr-policyfiles-managenodes
$ | chef show-policybase==== * acceptance: f21e2334b6* production: f21e2334b6
|
Now we can do move the node to the new policy group with the knife command as follows
Terminal: ~/lcr-policyfiles-managenodes
$ $ | knife node policy set mynode production baseSuccessfully set the policy on node mynode01knife node show mynodeNode Name: mynodePolicy Name: basePolicy Group: productionFQDN: ip-172-31-58-113.ec2.internalIP: 3.83.218.146Run List: recipe[base::default]Recipes: base::default, hardening::default, os-hardening::default, os-hardening::packages, os-hardening::yum, os-hardening::limits, os-hardening::login_defs, os-hardening::minimize_access, os-hardening::pam, os-hardening::profile, os-hardening::securetty, os-hardening::suid_sgid, os-hardening::sysctl, os-hardening::auditd, os-hardening::selinux, hardening::remediationPlatform: centos 7.6.1810Tags:
|
Policyfile Attributes
Our policyfile base specifies a value for the attribute default['base']['message'] that's defined in the base cookbook. This pattern works in simple use cases but can get complicated if you have multiple cookbooks in the run_list, each of them requiring attributes to be set.
However, it is normal to set attribute values for a specific environment, i.e. policy group. For example, a database used in 'acceptance' may be testdatabase, while in 'production' it is just database.
You can do this with policyfiles by using the 'policy group' name within the attribute definition in the policyfile as follows
Editor: Policyfile.rb
1
2
| default['acceptance']['myapplication']['database'] = 'testdatabase'
default['production']['myapplication']['database'] = 'database' |
Then you would use the following code within your recipe
Editor: recipes/default.rb
1
| database = node['myapplication']['database'] |
The correct attribute would then be provided based on the policy_group of the node. So, for example, with a policy_group of acceptance the attribute would contain testdatabase.
Update your policyfile policyfiles/base-policy.rb to use this technique, by including the policy group in the specific attribute values on lines 20-21 as follows,
Editor: policyfiles/base-policy.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| # Policyfile.rb - Describe how you want Chef to build your system.
#
# For more information on the Policyfile feature, visit
# https://docs.chef.io/policyfile.html
# A name that describes what the system you're building with Chef does.
name 'base'
# Where to find external cookbooks:
default_source :supermarket
# run_list: chef-client will run these recipes in the order specified.
run_list base::default'
# Specify a custom source for a single cookbook:
cookbook 'base', path: '../cookbooks/base'
cookbook 'hardening', path: '../cookbooks/hardening'
# Policyfile defined attributes
default['acceptance']['base']['message'] = "This node was hardened by Chef. Policyfile created for 'acceptance' at #{Time.now.utc}\n"
default['production']['base']['message'] = "This node was hardened by Chef. Policyfile created for 'production' at #{Time.now.utc}\n" |
We now need to recreate our policyfile lock file and reupload it to the Chef Server.
First remove the old base-policy.lock.json file and create the new version
Terminal: ~/lcr-policyfiles-managenodes
$$ | rm policyfiles/base-policy.lock.jsonchef install policyfiles/base-policy.rbBuilding policy baseExpanded run list: recipe[base::default]Caching Cookbooks...Installing base >= 0.0.0 from pathInstalling hardening >= 0.0.0 from pathUsing os-hardening 3.2.0Using windows-hardening 0.9.1Using windows-security-policy 0.3.9Using sysctl 1.0.5Using ohai 5.2.5 Lockfile written to /private/tmp/lcr-policyfiles-managenodes/policyfiles/base-policy.lock.jsonPolicy revision id: f21e2334b679f3b7a0580caa48dc74593fc19642f87349415ee645b1b12f82e4
|
View the existing policy on Chef Server (just so we can verify the change later)
Terminal: ~/lcr-policyfiles-managenodes
$ | chef show-policybase==== * acceptance: f21e2334b6* production: f21e2334b6
|
Upload the new policyfile
Terminal: ~/lcr-policyfiles-managenodes
$ | chef push production policyfiles/base-policy.rbUploading policy base (331312e659) to policy group productionUsing base 0.1.0 (beeb9200)Using hardening 0.1.0 (87f4601f)Using ohai 5.2.5 (f393ae21)Using os-hardening 3.2.0 (1255dd1c)Using sysctl 1.0.5 (3d2a2314)Using windows-hardening 0.9.1 (fb514509)Using windows-security-policy 0.3.9 (b2e3ba1e)
|
View the new policy on Chef Server to show it has indeed changed
Terminal: ~/lcr-policyfiles-managenodes
$ | chef show-policybase==== * acceptance: f21e2334b6* production: 331312e659
|
Now rerun chef-client on the node
Terminal: ~/lcr-policyfiles-managenodes
$ | knife ssh 'policy_group:production' -x nodeusername -P nodepassword 'sudo chef-client'...ec2-3-83-218-146.compute-1.amazonaws.com - update content in file /etc/motd from bf139a to 0c78edec2-3-83-218-146.compute-1.amazonaws.com --- /etc/motd 2019-02-11 11:02:50.683609092 +0000ec2-3-83-218-146.compute-1.amazonaws.com +++ /etc/.chef-motd20190211-16573-1camcth 2019-02-11 12:56:53.781926139 +0000ec2-3-83-218-146.compute-1.amazonaws.com @@ -1,2 +1,2 @@ec2-3-83-218-146.compute-1.amazonaws.com -This node was hardened by Chef. Policyfile created at 2019-02-11 10:33:30 UTCec2-3-83-218-146.compute-1.amazonaws.com +This node was hardened by Chef. Policyfile created for 'production' at 2019-02-11 12:48:04 UTC...ec2-3-83-218-146.compute-1.amazonaws.com Chef Client finished, 5/90 resources updated in 06 seconds
|
You'll notice the chef-client has picked up the appropriate attribute value from the policyfile.
The key takeaway here is we now have policy group (i.e. 'environment') level attributes in the policyfile, but we did not have to update our recipe.
| This deployment pattern of maintaining 'environment' level attributes within a policyfile may not be appropriate in all circumstances, especially if your workflow requires multiple policyfiles across your application stack, hence requiring you to duplicate values. Other approaches may be to use a wrapper cookbook to maintain attribute values based on Policy Group, or to nest Policyfiles. |
Using Policyfiles With Chef Zero
To this point we have seen how you can use policyfiles in conjunction with a Chef Server. However, a Chef Server is not mandatory, and your workflow may use Chef Solo instead. Also, you may be in an air gapped environment and may not have external access to GitHub or Chef Supermarket in order to pull down dependent files.
With policyfiles you can easily vendor all cookbook dependencies into a single artifact consisting of a chef-repo that can be transferred to your node.
The chef export command creates a Chef Zero compatible repository containing the cookbooks described in a Policyfile.lock.json. The exported repository also contains a .chef/config.rb which configures Chef to apply your policy locally.
| If you run the following command with '.' instead of './temp' then it will create the output locally BUT it will overwrite your existing .chef directory. |
Terminal: ~/lcr-policyfiles-managenodes
$ | chef export ./policyfiles/base-policy.rb ./tempExported policy 'base' to ./temp To converge this system with the exported policy, run: cd ./temp chef-client -z
|
Take a little while to explore the contents of this temp/ directory, in particularly the following
temp/README.mdtemp/cookbook_artifactstemp/.chef/config.rb
You can run that previous command with the -a in order to create the same contents within a tarball.
Terminal: ~/lcr-policyfiles-managenodes
$ | chef export ./policyfiles/base-policy.rb ./temp -aExported policy 'base' to lcr-policyfiles-managenodes/temp/base-f21e2334b679f3b7a0580caa48dc74593fc19642f87349415ee645b1b12f82e4.tgz
|
This tarball repo can be copied to the target machine and expanded, then you can apply the policy to the machine with chef-client -z.
Once you’ve generated the archive and transferred the file to your air-gapped environment, you could run the following command to load it up on the Chef Server
Terminal: ~/lcr-policyfiles-managenodes
$ | chef push-archive qa base-f21e2334b679f3b7a0580caa48dc74593fc19642f87349415ee645b1b12f82e4.tgz
|
Conclusion
Policyfiles give you a reliable way of ensuring you're using the correct version of artifacts at all times. A hash stored in the policyfile is checked against a hash of the content delivered to ensure Chef client is using the correct cookbooks. This content can be vendored and copied over to air gapped servers, or to servers running Chef Zero.
