Managing Google Cloud Platform
This topic explains how to get started with Google Cloud Platform (GCP) in Commander, as well as key concepts involved in understanding how Commander manages GCP. This topic assumes that you understand the basics of GCP as well as the basics of Commander.
In this topic:
To manage GCP services with Commander, here's how to get started:
- Create a service account for Commander to have programmatic access to GCP. See Create a service account for Commander to access GCP.
- Ensure that the proper APIs are enabled for the project where the service account was created. See Enable the required APIs.
- Give the service account permissions for all of the resources that you want to add as a single GCP managed system in Commander. See Grant permissions to the Commander service account.
- Optional: If Internet access is established through a web proxy server, integrate your web proxy server with Commander. See Connecting Public Clouds to Commander through a Web Proxy Server.
- Now you're ready to follow the steps in the Getting Started with Commander wizard.
In the GCP Console, you must create a new service account for Commander to have programmatic access to GCP as explained in this section.
GCP supports two types of authentication: service accounts and user accounts. Commander requires a service account.
To learn more about GCP service accounts, see Understanding Service Accounts in the GCP documentation.
- In the GCP Console, use the navigation menu to navigate to IAM & admin.
- In the header, select the project location for the new service account.
While the project location you choose has no impact on the service account's visibility and permissions, make sure to select a project that won't be deleted.
In this example, Embotics Project X has been selected as the location for the new service account.
- In the left menu, select Service accounts.
- On the Service accounts page, click Create Service Account.
- On the Service account details page, enter a distinct name, such as "Commander Service Account", optionally enter a description, and click Create.
The Service account ID field is automatically populated based on the name you enter.
- On the Grant this service account access to project (optional) page, click Continue. You will grant permissions in another context.
- On the Grant users access to this service account (optional) page, click Create Key.
- In the Create key (optional) section, keep the default key type, JSON, and click Create.
- If your browser prompts you to save the file, save it to a known location.
A JSON file that contains your key downloads to your computer. You'll use this JSON file when adding a GCP managed system.
This is the only time when you can download this private key.
- Click Done.
In the GCP Console, for the project where you created the service account for Commander, ensure that the following APIs are enabled:
- Cloud Resource Manager API — so that Commander can group resources by organization and project
- Cloud Billing API — so that Commander can retrieve billing data
To learn how to enable APIs for a project, see Enabling and Disabling Services in the Google Cloud documentation.
You must give the service account permissions for all of the resources that you want to add as a single GCP managed system in Commander. To learn more about GCP service accounts, see Understanding Service Accounts in the GCP documentation.
If you use Shared VPC networks, the Commander service account must have visibility of the host project.
- In a text editor, open the downloaded JSON file that contains your key.
- Copy the value for "client_email". In our example, the value is
- In the GCP Console, navigate to IAM & admin and select IAM.
- In the header, select a resource that you want to manage with Commander (such as an organization, folder or project).
In this example, the organization "embotics.com" is one of the resources we want to manage with Commander.
- On the IAM page, click Add.
- In the Add members pane, paste the "client_email" text you copied into the New members field.
- Click the Select a role drop-down list and set permissions as required.
- Assign the Project Editor permission so that Commander can carry out actions on resources within the selected object (such as powering instances on and off).
- If you selected an organization, assign the Folder: Folder Viewer and Organization: Organization Viewer permissions so that the Commander views display GCP resources in their proper structural hierarchy, rather than in a flat list.
- To monitor VM performance metrics, the service account must have at least the Monitoring Viewer role.
- Click Add Another Role and assign other permissions on this resource as necessary.
- Repeat steps 4 through 8 for each resource that you want to be part of this GCP managed system.
- Click Save.
What's next? Now you're ready to add a GCP managed system.
Commander provides three infrastructure views:
The Infrastructure view shows a hierarchy of the entire compute infrastructure and is designed for daily operations. Because this view groups resources by region and zone, it's the view you should use when thinking about geographic distribution. VM images and deployments aren't shown in this view.
By default, Commander displays only those GCP regions that contain resources. Displaying fewer regions improves performance by requiring less communication between Commander and GCP and by not requiring Commander to communicate with geographically distant regions. Fewer displayed regions also means improved usability for Commander administrators. To change what regions are displayed, see Selecting Public Cloud Regions to Display in Commander.
The Applications view shows a hierarchy of deployments, instances and images and is designed for provisioning new resources as well as managing deployments. This view provides an application-centric view of GCP resources, logically grouped by organization, folder and project.
The Storage view shows storage resources grouped by region. For GCP, all storage resources are displayed under the Managed folder. Both regional and zonal storage resources are organized into Commander datastores, which are logical groups for aggregating persistent VM disks. When you look at a datastore, you can see which VMs have persistent disks, as well as the total storage usage in that zone or region. Commander shows both regional and zonal storage resources.
The image below shows how Commander displays regional and zonal storage resources: us-central1 is Google Cloud Regional Persistent Storage, whereas us-central1-a, us-central1-b and so on are Google Cloud Zonal Persistent Storage.
Commander allows you to manage the GCP resources detailed in the following table.
|Infrastructure / Resource Type||Details|
GCP instances can have either a predefined instance type (such as f1-micro, n1-standard-1 or n1-highmem-2) or custom configuration of CPU and memory resources. Commander displays the instance type details for deployed instances.
Commander currently supports reconfiguring CPU and memory resources for a GCP instance manually, and through change requests. In addition to predefined GCP instance types, Commander supports custom values for memory and CPU, as well as extended memory. To learn more about custom instance types, see Creating a VM Instance with a Custom Machine Type in the GCP documentation. To learn more about extended memory, see Adding extended memory to a machine type in the GCP documentation.
Deployments can be managed from Commander's Applications view. All deployment resources (whether directly managed by Commander or not) are displayed on the deployment's Resources tab. To provision a deployment, you must first add a deployment configuration to the service catalog.
Both regional and zonal storage resources are organized into Commander datastores, which are logical groups for aggregating persistent VM disks. When you look at a datastore, you can see which VMs have persistent disks, as well as the total storage usage in that zone or region. Commander shows both regional and zonal storage resources.
Because GCP storage is elastic, Commander properties and variables related to capacity (for example, provisioning level) don't apply to GCP. Likewise, reports based on these properties don't include data for GCP (for example, the Over-provisioned Disk Summary Report).
During regular synchronization with GCP, Commander collects the storage used by persistent disks.
Images are equivalent to VM templates and are displayed as VM templates in Commander. GCP provides a set of public images. Any images you create in GCP are private images. Commander displays GCP images in the Applications view.
Each GCP project has a default virtual network, but may contain additional networks. Virtual networks are shown in Commander's Applications View, along with their subnets and the VMs within them. See also Networking and security.
Each GCP virtual network contains one or more subnets (although "legacy" networks don't). Subnets are shown in Commander's Applications View. You can assign network zones to subnets.
Organizations are the root node in the Cloud Platform Resource hierarchy. Commander's Applications View shows organizations, as well as the VMs, folders, virtual networks and subnets within them.
Folders are nodes in the Cloud Platform Resource Hierarchy. A folder can contain projects, other folders, or a combination of both. Commander's Applications View shows folders, as well as the organizations, VMs, virtual networks and subnets within them.
GCP projects form the basis for creating, enabling and using all GCP services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for GCP resources. Commander's Applications View shows projects, as well as the VMs, virtual networks and subnets within them.
A region is a specific geographical location where you can run your resources. Commander displays regions in the Infrastructure view or the Applications view and Storage views. See Selecting Public Cloud Regions to Display in Commander to learn which regions Commander displays by default, and how to change this.
Each region is divided into multiple zones. Zones are somewhat equivalent to on-premise datacenters. Resources that are located in a zone, such as instances or persistent disks, are referred to as zonal resources. Other resources, like static external IP addresses, are regional. Regional resources can be used by any resources in that region, regardless of zone, while zonal resources can only be used by other resources in the same zone. Commander's Infrastructure view or the Applications view View shows zones, as well as the VMs within them.
Changes made to a GCP managed system within Commander are displayed as soon as the task has finished, just as they are with on-premise managed systems. Unlike on-prem systems, however, Commander regularly retrieves changes made within GCP.
By default, Commander waits 60 minutes between updates from GCP. You can change this interval, but we recommend keeping at least 60 minutes between updates. More frequent updates (meaning lower values for this setting) may impact Commander performance, especially in large installations.
- When a service exists for a shorter time period than the update frequency, Commander may not discover the service during the update.
- An advanced system property can be used to define an update frequency of less than 10 minutes. Contact firstname.lastname@example.org to learn more.
- If you make changes outside Commander, and you want these changes to be visible in Commander before the next scheduled update, you can synchronize the inventory.
Two levels of service request automation support
Commander supports new service request automation for GCP deployments, and change request automation for both GCP deployments and VM instances. This means that:
- GCP deployment configurations can be added directly to the service catalog
- GCP deployments are provisioned directly by Commander
- GCP deployments are affected by approval and completion workflows for new service requests, as well as for change requests
By contrast, change request automation is supported for GCP VM instances, but new service request automation isn't. VM instances:
- can't be added directly to the service catalog; instead, they must be added as part of a GCP deployment configuration
- can be deployed only as part of a GCP deployment by Commander
- are affected by change request approval and completion workflows, but not by new service request workflows
It's important to understand this distinction. For example, when you add a deployment configuration to the service catalog, it becomes a service catalog component. A completion workflow can be assigned to the deployment configuration in the service catalog blueprint for execution on the provisioned deployment, but not to the resources that will be provisioned as part of the deployment.
Multi-cloud services aren't yet supported for GCP.
To learn more, see Adding GCP Services to the Catalog.
GCP mandates that deployment names comply with RFC 1035. Names must contain between 1 and 63 characters and must match the following regular expression:
Names must start with a letter, end with a letter or digit, and have as interior characters only letters, digits and hyphen.
GCP deployments must be provisioned into a project. Therefore, you must create a deployment destination for each project you want to target.
GCP instance types added between Commander releases are fully supported in the next Commander release.
If a GCP instance is found with an unsupported instance type, the instance type is displayed properly, but resource and cost information isn't available, and quota won't be calculated.
Commander displays all public cloud costs in US dollars (USD).
Commander uses hard-coded costs for all projected GCP costs, such as service catalog costs, deployed service costs and reports with a projected cost model configuration. Hard-coded costs are provided for all supported service types and instance types, by region.
To make sure your price list is current, you can use the command workflow "Update Public Cloud List Prices". The workflow checks whether your price list is current, then updates the price list if necessary. You can configure this workflow to run on a recurring schedule, so that your price list is always current. By default, this workflow is disabled, but you can set how frequently it runs. For more information, see Updating Public Cloud List Prices.
A cost model is automatically applied when a GCP account is added as a managed system. The cost model enables you to add IT support costs, custom costs like backup costs and application software licensing costs to GCP billing data and to Commander's hard-coded costs. You can create additional cost models for different parts of your public cloud account.
The cost model and cost files allow users to see accurate estimated GCP costs when viewing cost details for existing resources.
For shared-core, predefined and memory-optimized instances, which are charged by the unit, the instance cost is displayed.
For custom instances, where costs are based on CPU and memory resource usage, the individual CPU and memory costs are displayed.
GCP costing differs from AWS and Azure costing in an important way. GCP charges aggregated CPU and memory usage costs, grouped by project and service type; GCP billing data doesn't include costs for individual VMs. This means that:
- GCP doesn’t associate costs with VMs, the GCP VM Billing Report will always be empty. The GCP VM Comparative Economics Report will only show the projected cost model, not the historical or current cost models. For AWS and Azure, VM billing records and the VM Billing Report are based on retrieved billing data, so they will be populated as long as billing data retrieval is turned on.
- actual costs aren't displayed for GCP instances
- Commander doesn't retrieve labels from GCP billing data
- you may notice a few differences in the Cost Analytics Details tables when you're managing GCP. To learn more, see Troubleshooting Cost Analytics.
Commander doesn't support costs for:
- preemptible VMs (PVMs)
- sole-tenant nodes
- committed use discounts
- premium application licenses (SAP, SQL Server)
- disk snapshots
- network traffic
By default, Commander doesn't apply GCP's Sustained Use Discounts to estimated costs. To change this behavior, edit the advanced system property
embotics.cost.gcp.discount. After editing this system property, to trigger a cost recalculation, edit the GCP cost model and click Finish without making any changes.
GCP storage costs are per-region, but images don't have a region assignment. Commander displays image storage costs based on us-east1 region pricing.
To learn more about costing for GCP deployments, see also How Commander calculates costs for deployment configurations.
Commander supports quota for GCP, with the following exception: GCP deployments are not included in resource quota calculations. Resource quota-based service request approval workflows don't work for requested GCP deployments. However, because VMs are included in quota calculations, once a deployment is provisioned, any VMs in the deployment may cause quota to be exceeded. Cost quotas are fully supported for GCP services.
GCP's hybrid model for resource configuration means that Commander's rightsizing recommendations work differently for GCP than for AWS and Azure.
Aggressive rightsizing supported for custom machine types
GCP supports custom CPU and memory settings, like VMware and SCVMM. As a result, Commander's aggressive rightsizing feature, which recommends an immediate rightsizing to the most appropriate resource values, applies to GCP. It does not apply to AWS and Azure.
Recommendations always preserve current configuration
For a VM with a predefined instance type (such as n1-standard-1), Commander will always recommend another predefined instance type within the same family. As a result, if you've standardized the instance types in your environment, you can be sure that Commander will never recommend a custom machine type.
For a VM with a custom machine type, Commander will always recommend custom settings for CPU and memory. This ensures that if you were saving money through the use of custom machine types, Commander will never recommend a predefined instance type that could be much more costly.
Automatic conversion of custom to predefined types by GCP
If GCP finds an instance with a custom machine type that matches a predefined instance type, GCP automatically assigns the matching predefined instance type, thus affecting future rightsizing recommendations. For example, let's say you have an instance with a custom machine type of 12 vCPUs and 70 GB of memory. Because the instance is underused, Commander recommends aggressively downsizing to 8 vCPUs and 30 GB memory. Once you apply this recommendation, GCP automatically assigns the n1-standard-8 instance type, which has 8 vCPUs and 30 GB memory. From then on, Commander will always recommend another predefined instance type for this VM.
In addition, because recommendations don't consider sustained use discounts, the predicted cost may be less accurate towards the end of the month for an instance that was automatically assigned a predefined instance type.
Downsizing vCPU resources may increase memory costs
When the number of vCPUs is decreased, the amount of memory used by the VM may increase. As a result, Commander may sometimes predict little or no cost savings for a CPU downsizing recommendation, because the cost of the additional memory required may be the same as, or similar to, the savings in CPU costs.
GCP constraints may prevent recommendations
GCP rules governing memory, vCPUs and NICs can sometimes prevent Commander from making a recommendation.
- An instance's memory in GB can't be less than 90% of its current CPU count. Commander may not make a memory downsizing recommendation if the smaller memory size would break this rule.
- Commander won't make a recommendation that would result in the VM having an instance type that doesn't support the current number of NICs. An instance must have at least two and no more than eight NICs, and it may not have more NICs than vCPUs. For example, if an instance has eight NICs, Commander will never recommend downsizing the instance to have fewer than eight CPUs.
You can synchronize GCP labels, such as Cost Center, Business Unit, Product, Tier or Version, with Commander custom attributes. Like GCP, Commander uses key-value pairs to allow you to assign metadata to services and cloud infrastructure. Once assigned, this metadata persists throughout a service's lifecycle, enabling administrators to understand the purpose of each service. Label import provides:
- better targeting of power schedule recommendations — automatically set one power schedule for VMs with the label "dev" and another for those labeled "prod"
- advanced search and reporting — filter searches and reports by label, such as application ID or environment, or group report data by label
- workflow conditions based on label values — automatically select the right Chef recipe or Ansible playbook to run during post-provisioning, depending on compliance requirements
GCP instances can have multiple public and private IP addresses. In Commander, the IP Address property displays the first available public IP address; the Private IP property displays the first available private IP address. To view all assigned public and private IP addresses for a GCP instance, go to the instance's Summary tab, and on the General section, click Details next to the Private IP Address.
Commander also displays the number of NICs for each instance, and the IPv4 Addresses property (available in the Details section) provides a comma-separated list of all available IP addresses.
If you use Shared VPC networks, the Commander service account must have visibility of the host project.
To learn more about GCP networking, see Virtual Private Cloud (VPC) Network Overview in the GCP documentation.