Retrieving AWS Billing Data

You can configure vCommander to retrieve AWS billing data. Retrieving billing data improves the accuracy of costing reports and Cost Analytics, because these reports include metered usage costs such as Storage and Network I/O. Billing data is retrieved immediately after you configure the AWS report location, and during a nightly scheduled task, it's merged with the existing data in the VM billing records. Any credits in the billing data are also accounted for and will be displayed as a separate service type in the Cost Analytics page.

vCommander retrieves a maximum of 60 days' worth of billing data, and can retrieve data from before the AWS account was added as a managed system. vCommander also retrieves AWS billing data for VMs that have never been managed by vCommander. For example, if an AWS account was added as a managed system 90 days ago, and you configure vCommander to retrieve billing data today, vCommander retrieves the last 60 days' worth of billing data for all AWS VMs that have existed at any point over the last 60 days. As another example, if you add an AWS account as a managed system today, and you configure vCommander to retrieve billing data today, vCommander retrieves the last 60 days' worth of billing data for all AWS VMs that have existed at any point over the last 60 days, including VMs that were never managed by vCommander.

To learn more about how vCommander handles services it has not managed, see Billing data for all AWS services below.

AWS provides daily billing reports. To use Reserved Instance Recommendations and the Reserved Instance Planner Report, you must retrieve an hourly AWS billing report.

Consolidated AWS billing

vCommander supports AWS consolidated billing. A Payer Account is an account configured as the payer account in an AWS Consolidated Billing account family. A Linked Account is an account linked to a Payer Account in an AWS Consolidated Billing account family. vCommander's support for AWS consolidated billing is very flexible. You can retrieve billing data for:

  • a payer account and one or more linked accounts
  • a payer account on its own
  • a linked account on its own, as long as the linked account has access to the billing report on S3

See Consolidated Billing for Organizations in the AWS documentation for more information.

GovCloud billing

vCommander's supports the retrieval of GovCloud account billing data. AWS requires that a GovCloud account is associated with a standard AWS account for billing purposes. We call this standard account an Associated Account. To retrieve billing data for a GovCloud account, vCommander must also manage the Associated Account; both accounts must be added as managed systems.

See AWS GovCloud (US) Billing and Payment in the AWS documentation for more information.

Retrieving AWS billing data for a Payer Account

AWS billing data retrieval setup on the AWS side

Before you set up vCommander to retrieve billing records, you need to set up AWS to dump the billing records to an S3 bucket that you create. The general steps to do this are:

  1. Creating an S3 bucket
  2. Configuring permissions
  3. Creating an AWS billing report

Creating an S3 bucket

To create a bucket under the paying account where the billing reports will be delivered:

  1. Log in to AWS.
  2. Navigate to the S3 Management Console.
  3. Click Create bucket.
  4. Enter a name for your bucket in the Bucket name field. Your bucket name must contain between 3 and 63 characters and no spaces. You can use lowercase letters, numbers, hyphens and periods, but no underscores in your bucket name.
  5. Choose the Region for your S3 bucket. This is for the location only and doesn't impact the report data or vCommander.
  6. Click Create.

Configuring permissions

To set up permissions on the S3 bucket for AWS to be able to deliver the reports:

  1. In the Amazon S3 console, select the bucket that will receive your reports.
  2. In the Permissions tab, click Bucket Policy.
  3. Paste this text in the bucket policy editor:
    {
      "Version": "2012-10-17",
      "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "386209384616"
        },
        "Action": [
          "s3:GetBucketAcl",
          "s3:GetBucketPolicy"
        ],
        "Resource": "arn:aws:s3:::bucketname"
      },
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "386209384616"
        },
        "Action": "s3:PutObject",
        "Resource": "arn:aws:s3:::bucketname/*"
      }
      ]
    }
  4. Replace bucketname with the name of your bucket. Don't replace the Principal number 386209384616. AWS uses that account to deliver reports to the S3 bucket.
  5. Click Save.
  6. Adjust the vCommander AWS IAM role to have read permission on the S3 bucket and its content.

Creating an AWS billing report

To configure billing report generation:

  1. Go to the Billing and Cost Management Dashboard at https://console.aws.amazon.com/billing/.
  2. In the left-hand navigation list, click Reports.
  3. Click Create report.
  4. For Report name, type a name for your report.
  5. The Time unit is the time span over which you want the line items in the report to be aggregated. Hourly is recommended.
  6. Enable Resource IDs to include the individual resource IDs in the report.
  7. Click Next.
  8. Type the name of the S3 bucket you created in the Configuring permissions section.
  9. Click Verify to validate the bucket name and check that the bucket has appropriate permissions. See Configuring permissions for setting up permissions.
  10. For Report path prefix, type the prefix that you want to prepend to the name of your report. This creates an additional subdirectory for the report: <report-prefix>/<report-name>/yyyymmdd-yyyymmdd/<assemblyId>/<report-name>-<file-number>.csv.<zip|gz>. This is the path that you enter for Report Prefix in the AWS Billing Data Settings dialog.
  11. For Compression, select either .gzip or .zip. vCommander supports both compression methods.
  12. Click Next.
  13. Click Review and Complete.

    It might take up to 24 hours for your report from Amazon to be generated, so the Test button on the AWS Billing Data Settings dialog might not work right away.

vCommander billing data setup for a Payer Account

Access through:

Views menu > Operational or Deployed

Available to:

vCommander Roles of Superuser and Enterprise Admin; Administrator Access Rights

  1. Right-click an AWS managed system in the tree and select Retrieve Billing Data.
  2. If you use AWS consolidated billing: To retrieve billing data for a Linked Account, the Payer Account must also be configured for billing data retrieval. Likewise, to retrieve billing data for a GovCloud account, the Associated Account must also be configured for billing data retrieval. Therefore, it's most straightforward to start by selecting the Payer Account or Standard Account in the tree, because you can configure billing data retrieval for Linked and GovCloud accounts at the same time.

  3. In the AWS Billing Data Settings dialog, under Billing Report Location, select S3 Bucket.
  4. In S3 Bucket Name, enter the name of the S3 bucket that contains the billing report.
  5. Optional: If the report isn't in the root of the bucket, in Report Prefix, enter the report path prefix. Use forward slashes (/) to indicate subdirectories.
  6. In Report Name, enter the name of the report to retrieve.
  7. If you also want to retrieve billing data for one or more accounts linked to this Payer Account, under Consolidated Billing Accounts, selected the linked accounts.
  8. If you also want to retrieve billing data for a GovCloud account associated with this account, from Associated GovCloud Account, choose the appropriate GovCloud account.
  9. Be sure to associate the GovCloud account with the proper account.

    AWS Billing Data Settings
  10. Click Test to ensure that the report location is valid.

    If you see the message "Billing data found", the information entered is valid. Note that vCommander can't validate consolidated billing relationships. It might take up to 24 hours for your report from Amazon to be generated, so the Test button might not work right away. See also Troubleshooting below.

  11. Click OK.
  12. Billing data is retrieved immediately, and a nightly task to update billing data is scheduled.

Retrieving AWS billing data for a Linked Account

Access through:

Views menu > Operational or Deployed

Available to:

vCommander Roles of Superuser and Enterprise Admin; Administrator Access Rights

  1. Right-click an AWS managed system in the tree and select Retrieve Billing Data.
  2. If you use AWS consolidated billing: To retrieve billing data for a Linked Account, the Payer Account must also be configured for billing data retrieval. Therefore, it's most straightforward to start by selecting the Payer Account or Standard Account in the tree, because you can configure billing data retrieval for Linked and GovCloud accounts at the same time.

  3. In the AWS Billing Data Settings dialog, select Linked to Another Account, then choose the appropriate Payer account.
  4. AWS Billing Data Settings Linked
  5. If you also want to retrieve billing data for a GovCloud account associated with this account, from the Associated GovCloud Account list, choose the appropriate GovCloud account.
  6. Click OK.
  7. Sixty days of billing data is retrieved immediately, and a nightly task will update future records.

Retrieving AWS billing data for a GovCloud Account

Access through:

Views menu > Operational or Deployed

Available to:

vCommander Roles of Superuser and Enterprise Admin; Administrator Access Rights

  1. Right-click an AWS managed system in the tree and select Retrieve Billing Data.
  2. If you use AWS consolidated billing: To retrieve billing data for a GovCloud account, the Associated Account must also be configured for billing data retrieval. Therefore, it's most straightforward to start by selecting the Payer Account or Standard Account in the tree, because you can configure billing data retrieval for Linked and GovCloud accounts at the same time.

  3. Select Associated with Account, and select the appropriate GovCloud account from the list.
  4. Be sure to select the Associated Account. The Associated Account is not necessarily the Payer Account.

    AWS Billing Data Settings Associated Account
  5. Click OK.
  6. Sixty days of billing data is retrieved immediately, and a nightly task will update future records.

Disabling the retrieval of AWS billing data

To disable billing data retrieval for an AWS account:

  1. In the tree, right-click the AWS managed system for which you want to stop retrieving billing data and select Retrieve Billing Data.
  2. In the AWS Billing Data Settings dialog, select Not Configured and click OK.
  3. vCommander will no longer retrieve billing data for this managed system. If this account is a Payer Account, billing data retrieval is also disabled for any Linked Accounts. If this account has an associated GovCloud account, billing data retrieval is also disabled for the GovCloud account.

Troubleshooting

  • If you see the message "Billing data location is valid, but no data found," the selected account has access to the specified location, but the AWS billing report is not configured to use this location. Verify the billing report configuration in the AWS console.
  • If you see the message "Billing data location is invalid", verify the information in the AWS Console and correct the information as necessary.
  • If you see the message "Error occurred while validating Billing data location," check the vCommander log for details.
  • Even if you see an error message, you can click OK and address the problem later.

Billing data for VMs not managed by vCommander

vCommander also retrieves AWS billing data for VMs that have never been managed by vCommander, including charges for:

  • VMs that exist only between vCommander synchronizations with AWS, such as VMs spawned and torn down by an Auto Scaling Group
  • VMs that existed only before the AWS account was added as a vCommander managed system
  • VMs that existed only when the vCommander service was down
  • existing VMs, including for periods before the AWS account was added as a vCommander managed system

When a VM not managed by vCommander is discovered in the AWS billing report, the following occurs:

  • a cost model is applied to the VM, based on its location in the Operational or Deployed tree
  • ownership information is applied by the Default Ownership policy, or is inherited from the parent auto scaling group or stack

Because AWS doesn't include the VM's Virtual Private Cloud (VPC) in the billing report, any Default Ownership policies and cost models targeting VPCs in the Operational tree won't be applied to VMs discovered in the AWS billing report.

Note the following limitations:

  • The Default Attributes policy doesn't affect VMs discovered in the AWS billing report.
  • Volumes can't be mapped to VMs discovered in the AWS billing report, so storage costs are not included in the vCommander billing data for these VMs.

Billing data for all AWS services

When billing data retrieval is configured, vCommander's cost analytics data accurately reflects the total cost you'll have to pay on your AWS bill and includes costs for services that vCommander doesn't manage. Examples of such services include but are not limited to:

  • Amazon DynamoDB
  • Amazon Simple Storage Service
  • Amazon CloudWatch
  • Amazon ElastiCache
  • Amazon Route 53

On your AWS bill, some of the costs for services may be divided into sub-categories. These sub-categories don't show up in vCommander's cost analytics data. Only the total cost for a service is displayed.

Ownership of each service will reflect the ownership policy that you set up on the managed system, region, or availability zone level.

Monthly charges for AWS support plan fees are listed on the first day of each month. Tax isn't listed as a separate service because it's included in each individual service. For example, Virtual Machine has its own tax and RDS has its own tax.

If you configured billing data retrieval in release 6.x, billing for services not managed by vCommander will start as soon as you upgrade to release 7.x, but historical billing information won't be available.