Integrating Chef with vCommander

You can configure vCommander to use the Chef IT automation platform. Requested VMs can be automatically deployed and configured as Chef nodes. You can create workflows that specify the configuration for a node in the Chef server. You can also set up automatic VM decommissioning driven by expiry date or by a change request process, including removal of the node configuration from the Chef server.

You can integrate multiple Chef servers or multiple Chef organizations with vCommander, depending on how you're using Chef. This multi-tenant support extends throughout vCommander's automated provisioning, including the service catalog, request forms, workflows and workflow variables. For example, if you have one Chef organization for Linux and one for Windows, you can add both of these as separate integrations, so that vCommander is aware of the two distinct sets of users, cookbooks, recipes and roles. You can set the Chef organization in the service catalog blueprint, which means that you can easily configure the service definition for use by a specific tenant.

You can specify the Chef environment (such as development, testing, staging and production) for a component in the service catalog blueprint. Workflow variables allow you to return the specified environment. Both vCommander and Service Portal users can view the intended Chef environment for a Chef node, along with its roles and recipes.

Integrating vCommander with Chef allows you to:

set default values for environments and run-lists in the vCommander service catalog

optionally, present Chef configuration choices to end users when requesting new services

configure Chef-specific workflow steps for common functions such as creating nodes, bootstrapping nodes, supplying roles and recipes for node configuration, and deleting nodes during decommissioning

access VM configuration, metadata and request form data when configuring a node in the Chef server

identify Chef nodes and view their configuration in vCommander and the Service Portal

synchronize with the Chef server on a configurable schedule

What version of Chef is supported?

To set this up, you must use Chef 12.x (either hosted or installed).

Overview of tasks

The tasks involved in integrating with Chef are:

1.Add system credentials for the Chef server

2.Add a Chef server or Chef organization to vCommander

3.Configure Chef information in the service catalog

4.Optional: Allow users to set Chef roles and recipes on the request form

5.Optional: Allow users to set the Chef environment on the request form

6.Edit the example completion workflows for new requests

7.Set up a decommissioning workflow for Chef nodes

8.Optional: Adjust the schedule for Chef synchronization

Add system credentials for the Chef server or Chef organization

Access through:

Configuration menu > Credentials

Available to:

vCommander Role of Superuser and Enterprise Admin

Chef integration requires an RSA key in PEM format. When you create credentials for integrating Chef with vCommander, you must have a .PEM file to authenticate with.

Authentication between the Chef server and Chef clients

RSA public-key pairs are used to authenticate the chef-client with the Chef server every time a chef-client needs access to data that is stored on the Chef server. This prevents any node from accessing data that it shouldn't and ensures that only nodes that are properly registered with the Chef server can be managed. Each client has its own RSA key.

As shown in Edit the example completion workflows for new requests, when you use the Configure Chef workflow step to create a Chef node, vCommander creates a private RSA key for the client. You can then copy this key to the target VM in .PEM format with a Copy File workflow step.

For each Chef server or Chef organization you want to integrate with vCommander, follow this procedure:

1.On the Credentials page, click Add.

2.In the Add Credentials dialog, from Credential Type, choose RSA Key.

3.Enter the credentials for an account that vCommander can use to connect to the Chef server. You must use a .PEM file to authenticate, so you must paste the contents of the .PEM file into the RSA Key field.

4.Provide a description, like "Chef", to serve as a memory aid to administrators when configuring tasks requiring credentials.

5.From Category, choose System Credentials.

6.Click OK.

Add a Chef server or Chef organization to vCommander

Access through:

Configuration menu > System Configuration > Integration tab

Available to:

vCommander Role of Superuser

For each Chef server or Chef organization you want to integrate with vCommander, follow this procedure:

1.On the Integration page, click Add > Chef Server.

2.In the Host/IP field, type the host name or IP address and port number for the Chef server. The default port is 443.

You must use Chef 12.x (either installed or hosted).

3.From Credentials, choose the Chef credentials you created in the previous procedure.

4.The Node Name field allows vCommander to match any VMs currently managed by vCommander to nodes registered with the Chef server. The field is prepopulated with the variable #{target.dnsName}, which returns the DNS name of the VM. If you have a different naming convention for Chef nodes, enter it.

NotePencil-smallvCommander uses variable substitution to identify Chef nodes in your infrastructure. vCommander obtains a list of registered node names from the Chef server then compares them with VM names in your infrastructure. When there are matches, it marks the VM as a Chef node.
NotePencil-smallClicking vars-20x20 in any text field that supports variables opens the Variable Assistant, which allows you to select variables for the current context and provides help for each variable.

5.In the Organization field, type the name of the Chef organization you want to manage with vCommander.

6.The Display Name field is prepopulated with the value from the Organization field. If you want to use a different display name for this integration, enter it.

7.Click Test to test the connection.

vCommander tries to establish a connection to the specified server. This test does not verify the Node Name.

4.Once you see a Success message, click OK to save the configuration.

After you click OK, vCommander creates and runs the Chef Synchronization task. This task retrieves the currently configured environments, roles and recipes from the Chef server. It also matches any VMs currently managed by vCommander to nodes registered with the Chef server.

You can double-click the Chef Synchronization task in the Tasks tab at the bottom of the vCommander console to view the associated events. The following image shows that vCommander found Chef nodes, recipes, roles and environments:

Chef Synchronization

This task is scheduled to run nightly. See Synchronizing with Chef and Adjusting the Synchronization Schedule below for more information.

Configure Chef information in the service catalog

Access through:

Configuration menu > Service Request Configuration > Service Catalog tab

Available to:

vCommander Role of Superuser, Enterprise Admin

You can configure Chef information in the service catalog. You can then use variables to pass the default values to a completion workflow that instructs the Chef server to apply the configuration to requested VMs (as explained Edit the example completion workflows for new requests later in this topic).

If you want to allow users to specify Chef information on the request form (as discussed in Optional: Allow users to set a Chef run-list on the request form), the defaults you select in this procedure appear as defaults on the form.

The following are ideas of how you might set up your service catalog blueprint to match your requirements:

If you do not want to allow users to make changes on the form, configure the run-list on the Chef tab in the catalog blueprint, and do not add any Chef elements to the Form tab.

If your requesters are developers familiar with Chef, add the Chef Run-List control to the Form tab in the catalog, so that they can configure their own run-list. You can configure defaults on the Chef tab of the service catalog blueprint if you like.

If your requesters just need to specify one or more additional applications for a requested service, add the simpler Chef Roles or Chef Recipes elements to the form.

To configure Chef information in the service catalog:

1.Go to Configuration > Service Request Configuration > Service Catalog.

2.Locate the service in the list and click Edit.

3.Go to the component's page in the wizard. If you have integrated with Chef, you should see a Chef tab.

4.Click the Chef tab.

This page displays the Chef organizations, environments, roles and recipes that are retrieved from the Chef server.

5.If you have added multiple Chef servers or organizations, select a server or organization from the Chef Organization menu. Otherwise, the Chef organization is displayed as a read-only value.

6.Choose an environment from the Chef Environment menu.

The roles and recipes for the selected environment are displayed.

7.Select one or more default roles and recipes from the Available Roles and Recipes pick-list. Ctrl-click to select multiple roles and recipes. Use the arrow buttons between the lists to move your selections into the Current Run-List. Then, use the arrow buttons to the right of Current Run-List to order the roles and recipes properly.

NotePencil-smallIf you do not select default roles and recipes, no defaults are applied.

8.Click Finish. Or, if you want to allow users to configure roles and recipes when requesting this service, continue to the next section.

Optional: Allow users to set a Chef run-list on the request form

Access through:

Configuration menu > Service Request Configuration > Service Catalog tab

Available to:

vCommander Role of Superuser and Enterprise Admin

If you want to allow users to configure a Chef run-list or choose Chef roles and recipes when requesting a new service, edit the service catalog entry. In this example, we want users to specify the Chef run-list for new VMs. We can then use the run-list configured on the form as input to a VM completion workflow or command workflow (which we'll create next).

NotePencil-smallIf you want to customize the list of recipes or roles on the form (for example, if you want to display "friendlier" role and recipe names), you can create a list-type custom attribute for use on the form instead of using the Chef Recipes and Chef Roles form elements.
NotePencil-smallThe Chef Run-List form element can't co-exist with either the Chef Recipes or the Chef Roles form element.

1.Continuing from the previous section, in the Service Catalog blueprint, click the Form tab.

2.In the Toolbox at the right side of the page, under Chef, click the Chef Run-List form element.

The run-list is added as a service request form component.

Chef Form

3.In the newly added component, optionally enter a custom display label for the form element.

4.From the Selectable Values list, Ctrl-click all values you want to allow users to select.

5.Do one of the following:

If you want users requesting a service to be able to override the defaults you specify, enable Override Default Values.

If you do not want users requesting a service to be able to override the defaults you specify, clear Override Default

6.To force users to select one or more values, enable Required.

7.Click OK and Save.

8.If you want to predefine the Chef environment and/or allow requesters to specify the environment, continue to the next section (Optional: Allow users to set the Chef environment on the request form.

9.To test your form, go to Configuration > Service Request Configuration > Form Designer. Select the Default Service form in the list and click Preview. In the list of services, select the service you configured for Chef.

Optional: Allow users to set the Chef environment on the request form

Access through:

Configuration menu > Custom Attributes

Available to:

vCommander Roles of Superuser and Enterprise Administrator

The Configure Chef workflow step allows you to specify the Chef environment. If you want to allow requesters to select the Chef environment on the form, you need to create a list-type custom attribute and add it to the service catalog entry.

1.Continuing from the previous section, in the Service Catalog, click the Attributes tab.

2.Click Add Attributes.

3.At the bottom of the Add Attributes dialog, click Create New Attribute.

2.In the Configure Custom Attribute dialog, enter a name for the new custom attribute, for example, "Chef Environment".

4.From Type, choose List.

5.From Applies To, choose Form to create a custom attribute to capture information required only during the service request process (form attributes are not attached to deployed services).

6.Clear the Edit in Service Portal option, then click Next.

7.On the Configure Attribute page, enter a comma-separated list of values that users can select from a drop-down menu list for the attribute. For example:  "development,production,staging,testing", click Add, then Finish.

8.In the Add Attributes dialog, the new custom attribute is selected. Click OK.

9.On the Attributes tab, the new attribute is displayed. Optionally choose a default environment.

10.If you want to allow requesters to specify the Chef environment, click to the Form tab.

11.The new custom attribute is listed in the Toolbox at the right side of the tab. Click the new attribute to add it to the form.

12.Locate the attribute that was added to the Chef Environment form, and click Edit on the Chef Environment form element and enable the Required check box, if a value for this form element is required.

NotePencil-smallIf you did not specify a default environment on the Attributes tab, you should enable Required.

13.Click OK, and click Finish to save the catalog entry.

15.To test your form:

Go to Configuration > Service Request Configuration > Form Designer. Select the Default Service form in the list and click Preview. In the list of services, select the service you configured for Chef.

Or, log in to the Service Portal and request the Chef service you just configured.

Edit the example completion workflows for new requests

Access through:

Configuration menu > Service Request Configuration > Completion Workflow tab

Available to:

vCommander Role of Superuser and Enterprise Administrator

vCommander includes two example Chef workflows — one for Linux and one for Windows. These examples create the Chef node, install and configure the Chef client on the VM, and configure the Chef server. You must customize these example workflows. And once you've customized a completion workflow, you can copy it and make modifications for other operating systems.

Prerequisite: Ensure that the VM's networking and host name are properly configured, either through a customization spec or through workflow steps. For more information, including guidance on credentials, see Configuring OS Networking through a Workflow Step.

1.Go to Configuration > Service Request Configuration > Completion Workflow tab.

2.Select either the Chef for Linux Example or the Chef for Windows Example and click Edit.

3.On the Name page, customize the name if you want.

4.On the Steps page, the following steps are preconfigured:

Chef steps

5.The Wait for VM to Be Ready step simply waits for the service to obtain an IP address and DNS name.

Choose Add > Wait For Event, then do the following in the details pane:

From Organization, choose the appropriate option if you have integrated multiple Chef servers or organizations. Keep the default selection, Automatic, for a decommissioning workflow.

chef-comp-wf-wait
NotePencil-smallFor Windows VMs, it may take longer than the default 300 seconds (five minutes) to obtain an IP address and DNS name. You may want to use two steps: one that waits for guest OS customization to complete, and one that waits for IP address and DNS name.

6.The Create Node step creates a Chef node, using a run-list by default. This step also creates the private RSA key for the chef-client. We will access this key in a later step.

Choose Add > Guest OS > Configure Chef, then do the following in the details pane:

Action: By default, this step uses a run-list to configure the node. If you want to configure the node using a list of roles and recipes instead, select Create Node (Roles and Recipes) instead.

Node: By default, this field is prepopulated with the variable #{target.dnsName}. Edit this value if required.

NotePencil-smallvCommander uses variable substitution to identify Chef nodes in your infrastructure. vCommander obtains a list of registered node names from the Chef server then compares them with VM names in your infrastructure. When there are matches, it marks the VM as a Chef node.

Environment field: If you added the Chef environment to the request form as a custom attribute (as described above), enter the following variable in the Environment field:

#{target.settings.customAttribute['attribute name']}

For example, if you created a custom attribute called Chef Environment, enter the following variable:

#{target.settings.customAttribute['Chef Environment']}

Otherwise, leave the default value, #{target.settings.chef.environment}.

NotePencil-smallClicking vars-20x20 in any text field that supports variables opens the Variable Assistant, which allows you to select variables for the current context and provides help for each variable.

Recipes and Roles fields: By default, these fields are prepopulated with variables, to return the values selected on the form, or in the service catalog, if not selected on the form. Edit this value if required.

Run-List field: By default, this field is prepopulated with a variable that returns the run-list configured on the form, or in the service catalog, if not selected on the form. Edit this value if required. This field accepts a comma-separated list of roles and recipes, in Chef run-list format. For example:

role[linux],recipe[logrotate],recipe[logrotate::global]

The RSA key is stored at the following location, so that you can access it later if required for troubleshooting:

#{system.directory}\temp\#{target.chefNodeName}.key

where #{system.directory} is the location of the Tomcat directory in your vCommander installation.

chef-comp-wf-create-node

7.The Install Chef Client step is a comes a Run Program step with a custom title that will install the chef-client on the node.

Choose Add > Guest OS > Run Program, then do the following in the details pane:

Credentials: Select credentials from the drop-down menu or click Add Credentials. On Windows, Administrator credentials are required. See Workflow Steps Reference for credentials guidance.

In the Command Line field, enter the install command that you want to run.

Linux example

curl -L https://www.chef.io/chef/install.sh | bash -s -- && mkdir /etc/chef

Windows example

msiexec /qn /i https://opscode-omnibus-packages.s3.amazonaws.com/windows/2008r2/x86_64/chef-client-12.3.0-1.msi ADDLOCAL="ChefClientFeature"

chef-comp-wf-install-client

8.The Copy client.rb step loads the client.rb file, which is used to specify configuration details for the chef-client.

Choose Add > Guest OS > Copy File.

When you integrate a Chef server or organization, an example configuration file is saved to:

#{system.directory}\public\$org-client.rb

where #{system.directory} is the location of the Tomcat directory in your vCommander installation, and $org is the name of the Chef organization.

For example, if you integrated a Chef organization named "dev", the file would be located here:

C:\Program Files\Embotics\vCommander\tomcat\public\dev-client.rb

You can customize this configuration file. The default file looks similar to the following:

log_level        :auto

log_location     STDOUT

chef_server_url  "https://chef.example.com:443/organizations/dev"

ssl_verify_mode  :verify_none;

Do the following in the details pane:

Credentials: Choose the appropriate credentials from the list or click Add Credentials. See Workflow Steps Reference for credentials guidance.

Source: Specify the located of the client.rb file. Variables are used to specify the location of the client.rb file, as explained above:

#{system.directory}\public\#{target.settings.chef.organization}-client.rb

Destination: Customize the path to the chef-client installation on the target VM if necessary.

chef-comp-wf-client-rb

9.The Copy client.pem step uses the output from the Create Node step, the RSA private key, to create a .PEM file and copy it to the target VM:

Choose Add > Guest OS > Copy File, then do the following in the details pane:

Credentials: Choose the proper credentials from the menu. See Workflow Steps Reference for credentials guidance.

Source: The following path to the RSA key is prepopulated:
#{system.directory}\temp\#{target.chefNodeName}.key
where #{system.directory} is a variable that returns the location of the Tomcat directory in your vCommander installation.

Destination: Customize the path to the chef-client installation on the target VM if necessary.

chef-comp-wf-copy-pem

10.The final Run Chef Client step runs the chef-client:

Choose Add > Guest OS > Copy File, then do the following in the details pane:

Credentials: Select credentials from the  menu. See Workflow Steps Reference for credentials guidance.

Command Line for Linux: Customize the following if necessary:
chef-client -l error

Command Line for Windows: Customize the following if necessary:
c:\opscode\chef\bin\chef-client.bat -l error

chef-comp-wf-run-client

11.Click Next.

12.On the Assigned Components page, assign this workflow to the proper service catalog entries, then click Next.

Chef Completion Workflow Configuration Summary Page

12.On the Summary page, review the information and click Finish.

Workflow errors are written to the workflow step comment log. See Troubleshooting and Tracking Workflows for more information.

For more details on completion workflows, see Creating a Completion Workflow.

NotePencil-smallOnce you've set up one completion workflow, you can copy it and make modifications for other operating systems. Select the workflow on the Completion Workflow tab and click Copy.

Set up a decommissioning workflow for Chef nodes

Access through:

Configuration menu > Service Request Configuration > Completion Workflow tab

Available to:

vCommander Role of Superuser and Enterprise Administrator

This example configures the Chef server to delete the Chef node, powers off the node and deletes the node from disk. You can also create a command workflow to decommission a Chef node, using the same steps and actions.

1.Go to Configuration > Service Request Configuration > Completion Workflow tab and click Add.

2.On the Name page, provide a name such as "Decommission Chef Node", and from Apply this Workflow, choose after a Change Request is fulfilled, then click Next.

3.On the Steps page, we'll add three steps:

Delete Node

Power Off

Perform Remove Action

4.First we add a Configure Chef step with a Delete Node action.

NotePencil-smallThe Configure Chef step uses the credentials you created for integration with the Chef server.

Choose Add > Guest OS > Configure Chef, then in the details pane, do the following:

Optionally, change Step Name to "Delete Node".

From Organization, choose the appropriate option if you have integrated multiple Chef servers or organizations. Keep the default selection, Automatic, for a decommissioning workflow.

Delete Chef node

5.Next we add a Perform Power Action step with a Stop action that will execute only on running VMs.

Choose Add > Perform Power Action, then in the details pane, do the following:

From Step Execution, choose Execute when conditions are met, click Edit, and, in the Variable Assistant, enter the following, then click OK:

#{target.state} -eq "Running"

For more information on conditional steps, see Making Workflow Steps Conditional.

puppet-decom-wf-step2

6.Now we add a Perform Remove Action step with a Delete from Disk action.

Choose Add > Perform Remove Action, then in the details pane, in the Step Name field, type a step name of "Delete from Disk", then use the default settings.

puppet-decom-wf-step3

7.Click Next.

8.On the Assigned Forms page, select your Decommissioning form.

9.Click Next and Finish.

Now when a decommissioning change request is fulfilled, this workflow will run.

What happens next?

When a user requests the service you configured with Chef information, the VM will be automatically deployed and the completion workflow will run on the deployed VM. Once the Chef client on the deployed VM connects to the Chef server, the client will download its configuration and apply it to the VM.

Viewing Chef node configuration for a VM

Access through:

Views menu > Operational or VMs and Templates

Available to:

Administrator and All Operator Levels of Access Rights

When vCommander identifies a VM as a Chef node, a Chef tab is added to the Guest OS Details pane for the VM.

The name displayed on the Chef tab is the name of the node found on the Chef server.

Initially, the Chef tab contains no information. Click Refresh Chef Information to retrieve the intended role and recipe configuration for this node from the Chef server.

Chef tab

Once you have retrieved the configuration for a Chef node in vCommander, the information appears for Service Portal users. Service Portal users see the Refresh Chef Information button for VMs identified as Chef nodes as well.

Synchronizing with Chef and adjusting the synchronization schedule

Access through:

Tools > Scheduled Tasks

Available to:

All Access Rights Levels

Superuser can Override Schedules

When you integrate with a Chef server, vCommander creates and runs a Chef Synchronization task, to retrieve the currently configured roles and recipes from the Chef server as well as match any VMs currently managed by vCommander to nodes registered with the Chef server.

The task is scheduled to run nightly at 12:30 a.m. To synchronize manually:

1.Go to Tools > Scheduled Tasks.

2.Select the Chef Synchronization task and click Run Now.

Depending on how you use Chef and how much churn there is in your Chef environment, you may want to synchronize less frequently, synchronize once and then disable the task, or not synchronize at all.

To disable the task or change its schedule:

1.Go to Tools > Scheduled Tasks.

2.Select the Chef Synchronization task and click Edit.

3.To disable the scheduled task, uncheck the Enabled checkbox.

4.To change the schedule, edit the Frequency and/or the Recurrence.

NotePencil-smallIt's not possible to delete this scheduled task, but the task is automatically deleted if the Chef server integration is removed.

Disabling or removing a Chef server

Access through:

Configuration menu > System Configuration > Integration tab

Available to:

vCommander Role of Superuser

Disabling a Chef server makes the server unavailable for connections but saves all of your settings, meaning that you can return to the configuration dialog later and simply re-enable it. Disabling a server also disables the scheduled Chef synchronization task for that server. The Chef tab on the VM's Guest OS Details pane remains visible, but users will not be able to refresh Chef information. The settings on the Chef tab and the Form tab in the service catalog are also preserved.

Removing a Chef server clears the settings, meaning that you must reconfigure all of the settings if you want to reintegrate later. Removing a Chef server also removes the scheduled Chef synchronization task and the Chef tab on the VM's Guest OS Details pane.

To disable a Chef server

1.On the Integration page, locate the Chef server and click Edit.

2.Clear the Enabled checkbox and click OK.

To remove a Chef server

1.On the Integration page, locate the Chef server and click Remove.

2.Click Yes to confirm the change.