Cost Anomaly

Suggest Edits

This section guides you to configure the detection of cost anomalies in your cloud account using CoreStack.

Enabling Cost Anomaly for Cloud Accounts

CoreStack provides detailed insights about the cost anomalies identified in your cloud accounts. In order to fetch this information, the cost anomaly detector must be enabled for the cloud accounts in Account Governance dashboard. Cost anomalies can be detected only for the cloud accounts that has the cost anomaly detector enabled.

Perform the following steps to enable cost anomaly detector for a cloud account:

  1. Navigate to Account Governance submenu under Governance in the Left navigation menu.
  2. Click View option available in the Actions column of the required cloud account.
  3. Select View Settings from the menu. The Cloud Account Details screen appears.
  4. Select Governance Configuration from the left pane of the Cloud Account Details screen.
  5. Click Cost tab in the right side.
  6. Mark Enable Cost Anomaly checkbox.
  7. Click Save & Apply.

The cost anomaly detector will be enabled for the selected cloud account. These steps must be repeated for all the cloud accounts for which the cost anomaly detector must be enabled. Notification settings can be enabled for an email address or webhook, where notifications related to cost anomalies are sent.

Configuring Baseline Settings for Cost Anomaly

Once the cost anomaly detector has been enabled for required cloud accounts, baseline settings must be configured for the cost anomaly in CoreStack by performing the following steps:

  1. Click Settings optionand select Baseline Settings from the dropdown menu in the Cost Anomaly screen. The Cost Anomaly Baseline Settings screen will appear.
  2. Click Add New Settings. The Create Settings screen appears.
  3. Select the required boundary to define the area of influence for the cost anomaly settings: Tenant or Cloud Account.
    1. If Tenant is selected, select the required tenant from the Tenant dropdown list.
    2. If Cloud Account is selected:
      1. Select the required cloud provider from the Cloud Service dropdown list.
      2. Select the required cloud accounts listed in the Cloud Accounts dropdown list. Multiple cloud accounts can be selected.
  4. Select and add the required categories to be configured for the cost anomaly from the Add Categories dropdown list. The options are: Cloud Account, Resource Category, Tag, Product Category, and Resource Group. Multiple categories can be selected. A table will be displayed below with the selected categories.
  5. In the table, configure the following values for each category.
    1. Threshold: Specify the cost threshold (in percent value) to be set for a category for listing an anomaly under the specific category view in the Cost Anomaly screen.
    2. Anomaly Detection Span: Specify the frequency within which the cost anomaly must be considered for a category. The option are: Monthly, Quarterly, and Monthly&Quarterly.
    3. Category Count: Specify the count to be considered for a category.
  6. Specify a daily cost threshold (in percent value) limit in the Threshold compared to previous day field of Daily Threshold section, which will be used to evaluate if cost anomaly can be detected for a day.
  7. Specify a cost in the Minimum Cost Impact field of Daily Threshold section, which will be used to evaluate if cost anomaly exceeds this value when compared with the previous day.

Cost anomalies will be detected for a specific day only if the cost exceeds this threshold limit and the defined minimum cost impact in comparison with the previous day.

  1. Click Save.

The baseline settings for a cost anomaly will be configured successfully and listed in the Cost Anomaly Baseline Settings screen. CoreStack starts detecting the anomalies using the configured details. You can configure multiple cost anomaly baseline settings with different combinations and settings.

A cost anomaly baseline setting can be modified by clicking the Update icon available in the Action column of a baseline setting listed in the Cost Anomaly Baseline Settings screen.

Cost Anomaly Detection

The Cost Anomaly feature in CoreStack is used to detect any abnormal costs incurred in a cloud account. Anomalies can be detected if the daily spend is either too high or too low. Anomaly trends will be shared with users, who can take preventive actions accordingly.

A cost anomaly is detected in two ways:

  1. Through the billing data provided by the cloud provider.
  2. Through the event generated by the cloud provider when a cost spend event occurs.

Event-level anomalies can be identified within a few hours of their occurrence and, accordingly, remediation steps can be taken to mitigate the issue. Contrary to the event-level data, anomalies identified through billing data take approximately 24 - 48 hours to identify depending on the cloud provider, since there is a wait time involved with billing data being shared by the cloud provider.

Users can view the following three sections for Cost Anomaly in CoreStack:

  • Cost Anomaly (Billing) - Summary
  • Cost Anomaly (Event) - Summary
  • Cost Anomaly (Event) – List

After a cost anomaly is identified, an email notification is sent to the user as per the notification settings updated by them. The user can then take the required remediation steps from their end to manage their costs as per their preferences.

Cost Anomaly by Billing

There are different options available to view cost anomalies as per the available billing data. The Cost Anomaly (Billing) – Summary section displays the following graphs and bar charts:

  • Cost Impact by Cloud Provider
  • Cost Impact by Cloud Account
  • Cost Impact by Product Category
  • Cost Impact by Resource Category

Perform the following steps in CoreStack to view cost anomalies:

  1. Click Cost > Cost Anomaly.
  2. In the Tenant list, click to select a tenant.
  3. In the Cloud Provider list, click to select an appropriate provider option.
  4. In the Timeframe list, click to select the time period for which the cost anomaly needs to be viewed.
  5. In the Currency list, click to select an appropriate currency.

The Total Cost Impact, Total Anomalies, and Cost Impact by Cloud Provider for the selected cloud account will be displayed. These details are based on the billing information provided by the cloud provider.

  • Total Cost Impact: This shows the cost impact for multiple anomalies detected within a specific time period, and according to other filters chosen by the user.
  • Total Anomalies: This shows the total number of anomalies identified.
  • Cost Impact by Cloud Provider: This is a graph that shows the trend of cost impact within a set time period. For example, in the graph shown below, the cost impact trend for the month of March is displayed. Users can also see the cost impact amount for a particular date.

Users can also view detailed bar charts related to Cost Impact by Cloud Account, Cost Impact by Product Category, and Cost Impact by Resource Category. These bar charts are shown in descending order of cost impact – that is, the cloud account, product category, or resource category with the highest cost impact shows on the top and the one with lowest cost impact shows at the bottom.

📘

Note: A user can click any of the blue horizontal bars, select the appropriate option from the drop-down list, and then view the change in the shown graph/charts.

  1. On the Cost Impact by Cloud Provider graph, click any red dot (Cost Anomaly) to view more details. The Cost Anomaly – Root Cause section displays details about the root cause of an anomaly. User can also view the Cost Impact at Product & Resource Categories graph.
    In the Cost Impact at Product & Resource Categories graph, the product categories are listed on the left side and the resource categories are listed on the right side. A user can click any product category to view the list of resource categories related to it. Alternately, a user can click a resource category to view the product category mapped to it.
  1. To view the Cost Impact at Product & Resource Categories chart for a specific cloud account, product category, or resource category, a user can select from the appropriate options in the following fields:
    • Cloud Account
    • Product Category
    • Resource Category
      The Cost Impact at Product & Resource Categories chart will be displayed as per the options selected.

A detailed view of the above graph can be viewed in a tabular format as well. This format is called Cost Impact by Cloud Provider. The table shows the date when the event occurred, tenant name, cloud provider name, cloud account name, product category, resource category, resource name, and the cost impact amount.

Cost Anomaly by Event

There are different options available to view cost anomalies as per different events. The Cost Anomaly (Event) – Summary section shows the following:

  • Cost Impact by Cloud Provider
  • Cost Impact by Cloud Account
  • Cost Impact by Product Category
  • Cost Impact by Resource Category

Perform the following steps in CoreStack to view a cost anomaly by event:

  1. After a user performs steps 1 - 5 described above, they can view the Cost Anomaly (Event) – Summary section just below the Cost Anomaly (Billing) – Summary section.
    The Total Cost Impact, Total Anomalies, and Cost Impact by Cloud Provider for the selected cloud account should be displayed. These details are based on event data logged in the CoreStack application.
    • Total Cost Impact: This shows the cost impact for multiple anomalies detected within a specific time period, and according to other filters chosen by the user.
    • Total Anomalies: This shows the total number of anomalies.
    • Cost Impact by Cloud Provider: This is a graph that shows the trend for cost-spend within a set time period. In the graph shown below, the cost impact trend for the month of March is displayed. The cost impact amount for particular dates can also be viewed in this graph.

Users can also view detailed bar charts related to Cost Impact by Cloud Account, Cost Impact by Product Category, and Cost Impact by Resource Category. These bar charts are shown in descending order of cost impact – that is, the cloud account, product category, or resource category with the highest cost impact shows at the top while the one with lowest cost impact shows at the bottom.

📘

Note: A user can click any of the blue horizontal bars, select an appropriate option from the drop-down list, and view the change in the graphs/charts.

  1. On the Cost Impact by Cloud Provider graph, click any red dot (Cost Anomaly) to view more details. The Cost Anomaly – Root Cause section will be displayed. Users can also view the Cost Impact at Product & Resource Categories graph.
  2. To view the Cost Impact at Product & Resource Categories chart for a specific cloud account, product category, or resource category, a user can select from the appropriate options in the following fields:
    • Cloud Account
    • Product Category
    • Resource Category
      The Cost Impact at Product & Resource Categories chart will be displayed as per the options selected.

A detailed view of the above graph can also be viewed in tabular format. This format is called the Cost Impact by Cloud Provider – Detailed View. This table shows the date when the event occurred, tenant name, cloud provider name, cloud account name, product category, resource category, resource name, reason for the anomaly, and the cost impact amount.

Cost Anomaly by Event List

The Cost Anomaly (Event) - List section provides detailed anomaly information with an exact root cause to help users take remediation actions. The detailed information is shown in a tabular format. The table shows details related to the date when the anomaly was detected, tenant name, cloud provider name, cloud account name, product category, resource category, resource name, reason for anomaly, and cost impact amount.

After the user performs steps 1 - 5 described above, the Cost Anomaly (Event) – List section should be displayed at the end of the page.

Configuration for Cost Anomaly

Cost Anomaly settings can be configured at the account master level, tenant level, and cloud account level. Users can modify the sensitivity number and set it as per their needs in order to help detect anomalies.

The sensitivity number is crucial in deciding the upper and lower spending limits.

  • The daily average cost spend is derived by calculating the average cloud spend for the last 30 days.
  • The upper spend limit is the average daily spend plus the set sensitivity value.
  • The lower spend limit is the average daily spend minus the sensitivity value.

If the daily spend value goes above or below the upper and lower spend limit, then an anomaly is detected.

Perform the following steps to make configurations for the Cost Anomaly feature:

  1. Navigate to one of the following paths:
    • Account master level: Settings > Account > Cloud Account Settings > Cost
    • Tenant level: Settings > Tenants > Tenant > Cloud Account Settings > View Full Settings > Cost
    • Cloud account level: Account Governance > Select a Cloud Account > in the Actions column, right-click View > View Settings > Governance Configuration > Cost
  2. Click to move the slider to right to enable onboarding of Cost Anomaly to your account.
  3. Click Edit.
  1. In the Resource Category field, click the pen icon.
  2. Type the sensitivity number if it needs to be changed.

By default, the sensitivity is set to 30. If you want the sensitivity number to be changed, then you can feel free to modify it.

Cost Anomaly – Email Notifications

Overview

In the CoreStack application, the email notification settings for cost anomalies can be done at the account master level, tenant level, and cloud account level. An email notification is sent to users whenever a cost anomaly is detected – that is, when the cost spent value goes above or below the threshold value, then an anomaly is detected and an email notification is sent to users with all the relevant details.

The email notifications are sent within 24–48 hours of detection. Users can then take action accordingly based on the anomaly details provided to them.

📘

Note: For GCP accounts, the notifications are sent within 6-24 hours and for other accounts, the notifications are sent between 24-48 hours.

Configuring Cost Anomaly Email Notifications

Perform the following steps to set up cost anomaly email notifications:

  1. Navigate to one of the following paths:
    • For Account master level: Settings > Account > Cloud Account Settings > Cost > Cost Anomaly.
    • For Tenant level: Settings > Tenants > Tenants tab > select a tenant > Cloud Account Settings > View Full Settings > Cost tab > Cost Anomaly > select Email Address and Webhook, or any one option.
    • For Cloud account level: Governance > Account Governance > select a cloud account > in the Actions column, right-click View > View Settings > Governance Configuration > Cost tab > Cost Anomaly > select Enable Cost Anomaly.
  2. Click to move the slider to right to enable Cost Anomaly at account master or tenant levels.
  3. Click Edit.
  1. In the Email Addresses box, type the relevant email addresses of the recipients.
  2. In the Webhook box, type the URL of platforms where CoreStack integration is enabled.
  3. Click SAVE & CLOSE.

📘

Note: Refer to Cofiguration for Cost Anomaly procedure to know how to set sensitivity.

If an anomaly is detected, then a notification is sent to users. Refer to the sample notification email shown below. The anomaly details can be viewed in the email.

The email shows the date when the cost anomaly occurred, cost impact amount, cloud account type, cloud account name, cloud account ID, tenant name, product category, resource category, resource name (optional), daily average spend, expected spend range, cost spend, sensitivity percentage, etc.

Based on the cloud spend, four different types of notifications are sent:

  • Cloud Spend Above Threshold (based on last 30 days daily average spend)
  • Cloud Spend Below Threshold (based on last 30 days daily average spend)
  • Cloud Spend Above Threshold (based on yesterday's spend)
  • Cloud Spend Below Threshold (based on yesterday's spend)

Integrating Jira with Cost Anomalies

Jira can be configured with the cost anomalies created and actions can be triggered. In order to use Jira along with these cost anomalies, the Jira account must be onboarded into CoreStack. Refer Jira Integration topic for more details.

To configure Jira for triggering actions based on cost anomalies, perform the following steps:

  1. Click on the settings icon on the top left of CoreStack and select Tenants from the menu. Tenant Management screen appears.
  2. Click required tenant from the left side table. Configurations for the selected tenant will be displayed on the right side.
  3. Select Activity Queue Settings section.
  4. Enable Alerts checkbox and select the required Jira account from the dropdown list.
  5. Enable Policies checkbox and select the required Jira account from the dropdown list.
  6. Click Save icon.

The activity queue settings will be saved, and actions will be triggered in the associated Jira account for the cost anomalies based on the cost anomaly baseline setting created.

Navigation

After cost anomaly detectors are enabled for the required cloud accounts, relevant information and insights will be available in the following sections.

Click Cost in the Left navigation menu and select Cost Anomaly option to land in the Cost Anomaly screen. The Cost Anomaly screen provides a list of configured cost anomaly baseline settings, which can be viewed using different categories (refer the tabs explained below for categories and its views), along with cost overview such as total cost impact, total anomalies, peak anomaly day, peak anomaly account.

There will be 5 tabs available in the Cost Anomaly screen: Cloud Account, Resource Category, Tag, Product Category, and Resource Group.

  • Cloud Account: This view lists the cost anomaly baseline settings configured using cloud account category and provides a cost overview of the associated cost anomalies.
  • Resource Category: This view lists the cost anomaly baseline settings configured using resource category and provides a cost overview of the associated cost anomalies.
  • Tag: This view lists the cost anomaly baseline settings configured using tag category and provides a cost overview of the associated cost anomalies.
  • Product Category: This view lists the cost anomaly baseline settings configured using product category and provides a cost overview of the associated cost anomalies.
  • Resource Group: This view lists the cost anomaly baseline settings configured using resource group category and provides a cost overview of the associated cost anomalies.

Updated 6 months ago