Integrating Puppet with vCommander

You can configure Embotics® vCommander® to use the Puppet Labs® IT automation system, Puppet. When you integrate vCommander with Puppet as shown in this article, you can automate repetitive configuration tasks for VMs provisioned with vCommander.

How does vCommander work with Puppet?

vCommander integrates with Puppet using Puppet's REST API.

Requested VMs can be automatically deployed and configured as Puppet nodes. You can create workflows that specify the configuration for a node in the Puppet master. You can also set up automatic VM decommissioning driven by expiry date or a change request process, including removal of the node configuration from the Puppet master.

vCommander communicates only with the Puppet master. It does not communicate directly with Puppet agents, nor does it apply Puppet configuration to agents. But you can use a vCommander completion workflow to apply the configuration to the provisioned Puppet node immediately. Or, your completion workflow can simply configure the Puppet master, and leave the configuration of the newly provisioned VM to the Puppet infrastructure.

vCommander uses variable substitution to identify Puppet nodes in your infrastructure. For example, if your Puppet node name format is <dns_name>.pv.example.com, when you integrate with Puppet, you use #{target.dnsName}.pv.example.com to tell vCommander how to identify nodes. vCommander communicates with the Puppet master to obtain a list of registered node names and then compares them with VM names in your infrastructure. When vCommander finds a match, it marks the VM as a Puppet node.

Best Practice: Assign classes to nodes indirectly by assigning groups to nodes, rather than directly assigning classes to nodes. If you use the Configure Puppet workflow step to assign classes and variables to a node, vCommander creates a group with the same name as the node and pins the node to the group. A parent group named "vCommander" is also created to contain these groups.

Integrating vCommander with Puppet allows you to:

set the Puppet configuration in the vCommander service catalog

optionally, present Puppet configuration choices to end users when requesting new services

configure Puppet-specific workflow steps for common functions such as authorizing nodes and supplying the environment, groups and variables for node configuration

access VM configuration, metadata and request form data when configuring a node in the Puppet master

identify Puppet nodes and view their configuration in vCommander and the Service Portal

synchronize with the Chef server on a configurable schedule

NotePencil-smallvCommander does not display Puppet class parameters.
Authentication with the Puppet server

When vCommander executes commands on the Puppet Master via SSH, vCommander uses sudo if the user name of the credentials used to add the Puppet server is not "root". vCommander supports interactive sudo prompts.

What version of Puppet is supported?

vCommander Release 6.1.5 works with Puppet Enterprise 2015.2 or later, or Puppet Open Source 4.2 or later. If you have the Open Source edition, you must install and configure the Puppet Console on your Puppet Master so that vCommander can communicate with it.

Overview of tasks

The tasks involved in integrating with Puppet are:

1.Generate a certificate for the vCommander server

2.Create credentials

3.Add the Puppet server to vCommander

4.Configure Puppet information for service catalog components

5.Optional: Allow users to specify Puppet groups on the request form

6.Edit the example completion workflow for new requests

7.Set up a decommissioning workflow for Puppet nodes

8.Optional: Adjust the schedule for Puppet synchronization

Generate a certificate for the vCommander server

To allow vCommander to communicate with Puppet, you must generate a certificate for the vCommander server.

For more information on how vCommander communicates with Puppet, see Forming Node Classifier Requests in the Puppet documentation.

NotePencil-smallBy default, configuration data is stored at /etc/puppetlabs on the Puppet server. If your configuration data is stored elsewhere, adjust the file paths in this procedure.

1.Open an SSH connection to the Puppet server.

2.Generate a certificate for the vCommander server with the following command:

puppet cert --generate <vCommander hostname>

3.To add the vCommander server to the whitelist, add a new line containing the vCommander host name to the following file:

/etc/puppetlabs/console-services/rbac-certificate-whitelist

4.Restart the Puppet service pe-console-services for the whitelist change to take effect:

sudo service pe-console-services restart

5.Make a copy of the following files, which you need when integrating Puppet with vCommander:

Puppet's CA certificate:
/etc/puppetlabs/puppet/ssl/certs/ca.pem

The vCommander you just generated:
/etc/puppetlabs/puppet/ssl/certs/<vCommander_hostname>.pem

The vCommander private key you just generated:
/etc/puppetlabs/puppet/ssl/private_keys/<vCommander_hostname>.pem

Create credentials

Access through:

Configuration menu > Credentials

Available to:

vCommander Role of Superuser and Enterprise Admin

You need to create four sets of credentials in the System category:

credentials for an account that vCommander can use to open an SSH connection to the Puppet server

credentials to store the Puppet CA certificate

credentials to store the vCommander certificate you created above

credentials to store the vCommander private key you created above

Create credentials for vCommander to SSH to the Puppet server

1.Go to Configuration > Credentials.

2.On the Credentials page, click Add.

3.Leave the default Credential Type, Username/Password.

4.Enter the user name and password for an account that vCommander can use to open an SSH connection to the Puppet server.

The vCommander integration with Puppet automatically uses sudo with non-root-user credentials, so the user must be able to run sudo in this case. vCommander supports interactive sudo prompts.

5.For the description, enter "Puppet", to serve as a memory aid to administrators when configuring tasks requiring credentials.

6.From Category, choose System Credentials.

7.Click OK.

Create credentials to store the Puppet CA certificate

1.Go to Configuration > Credentials.

2.On the Credentials page, click Add.

3.In the Add Credentials dialog, from Credential Type, choose RSA Key.

4.In the Username field, enter a descriptive label for this certificate (for example, "Puppet CA Certificate").

5.In the RSA Key field, paste the entire contents of the file you copied from /etc/puppetlabs/puppet/ssl/certs/ca.pem.

6.Optionally, enter a description.

7.From Category, choose System Credentials.

8.Click OK.

Create credentials to store the vCommander certificate

1.Go to Configuration > Credentials.

2.On the Credentials page, click Add.

3.For Credential Type, choose RSA Key.

4.In the Username field, enter a descriptive label for this certificate (for example, vCommander Certificate).

5.In the RSA Key field, paste the entire contents of the file you copied from /etc/puppetlabs/puppet/ssl/certs/<vCommander_hostname>.pem and optionally enter a description.

6.For Category, choose System Credentials

7.Click OK.

Create credentials to store the vCommander private key

1.Go to Configuration > Credentials.

2.On the Credentials page, click Add.

3.For Credential Type, choose RSA Key.

4.In the Username field, enter a descriptive label for this certificate (for example, vCommander Private Key).

5.In the RSA Key field, paste the entire contents of the file you copied from /etc/puppetlabs/puppet/ssl/private_keys/<vCommander_hostname>.pem.

Optionally enter a description.

6.For Category, choose System Credentials

7.Click OK.

Add the Puppet server to vCommander

Access through:

Configuration menu > System Configuration > Integration tab

Available to:

vCommander Role of Superuser

1.Go to Configuration > System Configuration > Integration tab.

2.On the Integration page, click Add > Puppet Server.

3.In the Puppet Server dialog, in the SSH Host/IP field, enter the host name or IP address and port number for the Puppet server. The default port is 22.

All commands are executed by opening an SSH connection on the Puppet master, executing the command, gathering the output and terminating the connection.

4.From SSH Credentials, choose the Puppet credentials you created above.

5.The Puppet API Host/IP field is auto-populated with what you entered for the SSH Host/IP field. Edit this value if required.

6.From Puppet CA Certificate, choose the Puppet CA certificate credentials you created above.

7.From vCommander Certificate, choose the vCommander certificate credentials you created above.

8.From vCommander Private Key, choose the vCommander private key credentials you created above.

9.The Node Name field allows vCommander to match any VMs currently managed by vCommander to nodes registered with the Puppet master. The field is prepopulated with the variable #{target.dnsName}, which returns the DNS name of the VM. If you have a different naming convention for Puppet nodes, enter it.

NotePencil-smallClicking vars-20x20 in any text field that supports variables opens the Variable Assistant, which allows you to select variables for the current context and provides help for each variable.

10.Click Test to test the connection.

vCommander tries to establish an SSH connection to the specified server and look up the default group on the node classifier. This test does not verify the Node Name.

11.Once you see a Success message, click OK to save the configuration.

After you click OK, vCommander creates and runs the Puppet Synchronization task. This task retrieves the currently configured node groups and node classes from the Puppet server. It also matches any VMs currently managed by vCommander to nodes registered with the Puppet master.

You can double-click the Puppet Synchronization task in the Tasks tab at the bottom of the vCommander console to view the associated events. Or, go to the Events tab. The following image shows that vCommander found Puppet classes, groups, and environments:

Puppet Synchronization

This task is scheduled to run nightly. See Synchronizing with Puppet and Adjusting the Synchronization Schedule below for more information.

Configure Puppet information for service catalog components

Access through:

Configuration menu > Service Request Configuration > Service Catalog tab

Available to:

vCommander Role of Superuser, Enterprise Admin

You can predefine the Puppet environment as well as Puppet classes and groups in the service catalog. You can then use variables to pass these values to a completion workflow that instructs the Puppet master to apply the configuration to requested VMs (as explained later in this topic).

Best Practice: Assign classes to nodes indirectly by assigning groups to nodes, rather than directly assigning classes to nodes. If you use the Configure Puppet workflow step to assign classes and variables to a node, vCommander creates a group with the same name as the node and pins the node to the group. A parent group named "vCommander" is also created to contain these groups.
NotePencil-smallIf you have upgraded from vCommander version 5.2 or earlier, predefining Puppet classes and groups in the service catalog requires you to use the Blueprint catalog model.

If you want to allow users to specify Puppet groups on the request form (as shown in the next section), the values you select appear as default values on the form.

1.Go to Configuration > Service Request Configuration > Service Catalog tab.

2.Locate the service in the list and click Edit.

3.Go to the component's page in the wizard.  If you have integrated with Puppet, you should see Puppet tab.

4.Click the component's Puppet tab. All environments, classes and groups retrieved from the Puppet server are displayed.

Puppet tab

5.Choose an environment from those listed. The default environment is production.

NotePencil-smallIf the production environment does not exist and you do not select another environment, class and group selections will have no effect.

Once you choose an environment, only those classes and groups found in that environment are available for selection.

6.As required, select one or more classes and groups from the pick lists. Ctrl-click to select multiple classes and groups. If you do not select classes or groups, no default values are applied.

Best Practice: Assign classes to nodes indirectly by assigning groups to nodes, rather than directly assigning classes to nodes.

7.If you want to allow users to select Puppet groups when requesting this service, continue to the next section. Otherwise, click Finish.

Optional: Allow users to specify Puppet groups on the request form

Access through:

Configuration menu > Service Request Configuration > Service Catalog tab

Available to:

vCommander Role of Superuser, Enterprise Admin

If you want to allow users to specify Puppet groups when requesting a new service, edit the service catalog entry. In this example, we want users to specify Puppet groups for new VMs. We can then use the groups selected on the form as input to a VM completion workflow or command workflow (which we'll create next).

Best Practice: Allow users to select Puppet groups, rather than individual classes, on the form.

NotePencil-smallIf you want to customize the list of groups on the form, instead of using the Puppet Groups form elements, you can create a list-type custom attribute for use on the form instead.
NotePencil-smallYou can also use a list-type custom attribute to allow users to select an environment on the form, but this only makes sense if groups are the same across all environments.

1.Continuing from the previous procedure, in the Service Catalog entry, click the Form Designer tab.

2.In the Toolbox at the right side of the page, under Puppet, click the Puppet Groups form element.

Puppet Form Designer

3.In the newly added component, optionally enter a custom display label for the form element.

4.From the Selectable Values list, Ctrl-click all values you want to allow users to select.

5.If you want the values users select to be merged with the defaults you specified, disable Override Default Values. If you want users to be able to override the defaults you selected on the Puppet tab, enable Override Default Values.

If you want users requesting a service to be able to override the defaults you specify, enable Override Default Values.

If you do not want users requesting a service to be able to override the defaults you specify, clear Override Default

6.To force users to select one or more values, enable Required.

NotePencil-smallWhen this element is required, a user must select at least one value, even if you have already configured default values.

7.To allow users to select multiple values, enable Select Multiple.

8.Click OK and Save.

9.To test your form, go to Configuration > Service Request Configuration > Form Designer. Select the Default Service form in the list and click Preview. In the list of services, select the service you configured for Puppet.

Edit the example completion workflow for new requests

Access through:

Configuration menu > Service Request Configuration > Completion Workflow tab

Available to:

vCommander Role of Superuser and Enterprise Administrator

vCommander includes two example Puppet workflows.

If you use rules to set node groups, or if you use Hiera, customize the Puppet for Linux Basic Example workflow. This workflow simply installs the Puppet agent and authorizes the node.

If you want to preassign the environment, groups or classes to the new node for the first communication between the Puppet master and the agent, customize the Puppet for Linux Advanced Example workflow as shown in this section. This workflow creates the Puppet node, installs and configures the Puppet agent on the VM, and configures the Puppet master.

To use either of the example workflows, you must add Guest OS credentials. See Workflow Steps Reference for credentials guidance. Once you've customized a completion workflow, you can copy it and make modifications for other operating systems.

Prerequisite: Ensure that the VM's networking and host name are properly configured, either through a customization spec or through workflow steps. For more information, including guidance on credentials, see Configuring OS Networking through a Workflow Step.

Notes

For Windows VMs, it may take longer than the default 300 seconds (five minutes) to obtain an IP address and DNS name. You may want to use two steps: one that waits for guest OS customization to complete, and one that waits for IP address and DNS name.

The Authorize Node action of the Configure Puppet step authorizes communication between the Puppet master and the Puppet node. This action is not required if the Puppet master is configured to authorize all new nodes, because the master will authorize communication as soon as the agent connects to it.

To customize the Puppet for Linux Advanced Example workflow:

1.Go to Configuration > Service Request Configuration.

2.Click the Completion Workflow tab, select the workflow and click Edit.

3.On the Name page, customize the name if you want.

4.On the Steps page, the following steps are preconfigured:

5.The Wait for VM to Be Ready step simply waits for the service to obtain an IP address and DNS name.

Choose Add > Wait For Event, then do the following in the details pane:

Organization: choose the appropriate option if you have integrated multiple Chef servers or organizations. Keep the default selection, Automatic, for a decommissioning workflow.

Wait for VM to Be Ready step
NotePencil-smallFor Windows VMs, it may take longer than the default 300 seconds (five minutes) to obtain an IP address and DNS name. You may want to use two steps: one that waits for guest OS customization to complete, and one that waits for IP address and DNS name.

6.The Install Puppet Client step installs the Puppet agent on the new VM. Note that if the agent has already been installed on the source template, the template just needs to be configured to use the new node name.

Choose Add > Guest OS > Run Program, then do the following in the details pane:

Credentials: Choose the appropriate credentials or click Add Credentials. See Workflow Steps Reference for credentials guidance.

Command Line: Enter the following:

"curl -k https://#{integrations.puppet.address}:8140/packages/current/install.bash | bash"

NotePencil-smallClicking vars-20x20 in any text field that supports variables opens the Variable Assistant, which allows you to select variables for the current context and provides help for each variable.
Install Puppet Client step

7.The Copy puppet.conf step configures the Puppet agent:

Choose Add > Guest OS > Copy File.

When you integrate a Puppet server, an example configuration file is saved to:

#{system.directory}\public\puppet.conf

where #{system.directory} is the location of the Tomcat directory in your vCommander installation.

You can customize this configuration file. The default file looks similar to the following:

[main]

 logdir = /var/log/puppet

 rundir = /var/run/puppet

 server = pe.example.com

[agent]

 report = true

 classfile = $vardir/classes.txt

 localconfig = $vardir/localconfig

 pluginsync = true

The server line in the configuration file references the Puppet master server.

Do the following in the details pane:

Credentials: Choose the appropriate credentials from the list or click Add Credentials. See Workflow Steps Reference for credentials guidance.

Destination: customize the path to the Puppet agent installation on the target VM if necessary.

Copy File to Guest step

8.The Start Puppet Agent step comes next:

Choose Add > Guest OS > Run Program, then do the following in the details pane:

Credentials: Select credentials from the menu. The Configure Puppet step uses the credentials you created for communications with the Puppet server. See Workflow Steps Reference for credentials guidance.

Command Line: Customize the following entry, if necessary: "service puppet start"

Start Puppet Agent dialog

9.Next come three instances of the Configure Puppet step. For the first instance, you will configure an Authorize Node action, to authorize communication between the Puppet master and the Puppet node. This step is not required if the Puppet master is configured to authorize all new nodes, because the master will authorize communication as soon as the agent connects to it.

The first time the agent connects to the Puppet master, if the node has not been created, it is created automatically, and the default environment (Production) plus any default classes and groups are assigned to it. You only need to perform the set environment or group steps if you want to preassign the environment or groups to the new node for the first communication between the Puppet master and the agent.

Choose Add > Guest OS > Configure Puppet, then add the following entries in the details pane:

Authorize Node step

10.The Set Environment step sets an environment in the Puppet master for this VM.

Choose Add > Guest OS > Configure Puppet, then add the following entries in the details pane:

The Node field is prepopulated with the value entered for Node Name during integration with Puppet.

We can enter either comma-separated values or vCommander variables for the environment. Since we specified default values for Puppet Environment in the service catalog entry, we can retrieve this information with the following variables:

#{target.settings.puppet.environment}

Puppet Set Environment step

11.The Set Group step sets the node groups this node belongs to.

Choose Add > Guest OS > Configure Puppet, then add the following entries in the details pane:

We can enter either comma-separated values or vCommander variables for groups. Since we specified default values for Puppet Group in the service catalog entry and we added Puppet Groups to the request form, we can retrieve this information with the following variables:

#{target.settings.puppet.groups}

Puppet Set Groups step

12.Click Next.

13.On the Assigned Components page, assign this workflow to the proper service catalog entries.

Assigned Components

14.Review the information on the Summary page and click Finish.

Workflow errors are written to the workflow step comment log. See Troubleshooting and Tracking Workflows for more information.

For more details on completion workflows, see Creating a Completion Workflow.

Set up a decommissioning workflow for Puppet nodes

Access through:

Configuration menu > Service Request Configuration > Completion Workflow tab

Available to:

vCommander Role of Superuser and Enterprise Administrator

This example configures the Puppet master to delete the Puppet node, powers off the node and deletes the node from disk.

1.Go to Configuration > Service Request Configuration > Completion Workflow tab and click Add.

2.On the Name page, provide a name such as "Decommission Puppet Node", and from Apply this Workflow, choose after a Change Request is fulfilled, then click Next.

3.On the Steps page, we'll add the following three steps:

Delete Node

Power Off

Perform Remove Action

4.First we add an an Execute SSH Command step to configure the Puppet master to delete the node.

Choose Add > Execute SSH Command, then in the details pane, do the following:

In the Step Name field, optionally, change the name to "Delete Node".

From Credentials, choose the proper credentials. This step requires Guest OS credentials, so you must create a copy of the credentials used to integrate with Puppet, which are System credentials.  

In the Host Name field, enter the host name of the Puppet server.

In the Command Line field, type the following: sudo puppet node purge #{target.dnsName}

Delete Node step

5.Next we add a Perform Power Action step with a Stop action that will execute only on running VMs.

Choose Add > Perform Remove Action, then in the details pane, do the following:

In the Step Name field, optionally, change the name to "Power Off".

From Step Execution, choose Execute when conditions are met, click Edit, and, in the Variable Assistant, enter the following, then click OK:

#{target.state} -eq "Running"

For more information on conditional steps, see Making Workflow Steps Conditional.

puppet-decom-wf-step2

6.Now we add a Perform Remove Action step with a Delete from Disk action.

Choose Add > Perform Remove Action, then in the details pane, in the Step Name field, type a step name of "Delete from Disk", then use the default settings.

puppet-decom-wf-step3

7.Click Next.

8.On the Assigned Forms page, select your Decommissioning form.

9.Click Next and Finish.

Now when a decommissioning change request is fulfilled, this workflow will run.

What happens next?

At this point, you have two options:

Add an Execute SSH Command step to your Puppet completion workflow, with a command line that applies the configuration to the Puppet node immediately.

Wait for the Puppet agent to connect to the Puppet master, at which point the agent will download its configuration and apply it.

Viewing Puppet node configuration for a VM

Access through:

Views menu > Operational or VMs and Templates

Available to:

Administrator and All Operator Levels of Access Rights

When vCommander identifies a VM as a Puppet node, a Puppet tab is added to the Guest OS Details pane for the VM.

The name displayed on the Puppet tab is the name of the node found on the Puppet master.

Initially, the Puppet tab contains no information. Click Refresh Puppet Information to retrieve the intended environment, class, group and variable configuration for this node from the Puppet master.

Puppet tab

Once you have retrieved the configuration for a Puppet node in vCommander, the information appears for Service Portal users. Service Portal users see the Refresh Puppet Information button for VMs identified as Puppet nodes as well.

Synchronizing with Puppet and adjusting the synchronization schedule

Access through:

Tools > Scheduled Tasks

Available to:

All Access Rights Levels

Superuser can Override Schedules

When you integrate with a Puppet server, vCommander creates and runs a Puppet Synchronization task, to retrieve the currently configured environment, groups, classes and variables from the Puppet server, as well as match any VMs currently managed by vCommander to nodes registered with the Puppet master.

The task is scheduled to run nightly at 12:30 a.m. To synchronize manually:

1.Go to Tools > Scheduled Tasks.

2.Select the Puppet Synchronization task and click Run Now.

Depending on how you use Puppet and how much churn there is in your Puppet environment, you may want to synchronize less frequently, synchronize once and then disable the task, or not synchronize at all.

To disable the task or change its schedule:

1.Go to Tools > Scheduled Tasks.

2.Select the Puppet Synchronization task and click Edit.

3.To disable the scheduled task, clear the Enabled checkbox.

4.To change the schedule, edit the Frequency and/or the Recurrence.

NotePencil-smallIt's not possible to delete this scheduled task, but the task is automatically deleted if the Puppet server integration is removed.

Disabling or removing the Puppet server

Access through:

Configuration menu > System Configuration > Integration tab

Available to:

vCommander Role of Superuser

Disabling the Puppet server makes the server unavailable for connections but saves the settings, meaning that you can return to the configuration dialog later and simply re-enable it. Disabling the server also disables the scheduled Puppet synchronization task. The Puppet tab on the VM's Guest OS Details pane remains visible, but users will not be able to refresh Puppet information. The settings on the Puppet tab and the Form tab in the service catalog are also preserved.

Removing the Puppet server clears the settings, meaning that you must reconfigure all of the settings if you want to reintegrate later. Removing the Puppet server also removes the scheduled Puppet synchronization task and the Puppet tab on the VM's Guest OS Details pane.

To disable the Puppet server

1.On the Integration page, locate the Puppet server and click Edit.

2.Clear the Enabled checkbox and click OK.

To remove the Puppet server

1.On the Integration page, locate the Puppet server and click Remove.

2.Click Yes to confirm the change.