Anomaly Detector

Being able to monitor and detect abnormalities in your cloud resources is important because it helps you provide highly available and reliable services.

This section guides you to identify details about the anomalies observed in your cloud account using CoreStack, even if they are rare in occurrence, for all the cloud resources.

Configuring Anomaly Detector

To discover and fetch information about the anomalies in cloud resources, an anomaly detector must be configured and integrated with CoreStack. During its integration, the anomaly detector can be set up for anomaly detection for all the required cloud accounts that are onboarded into CoreStack.

Currently the Azure Anomaly Detector is supported for integration with CoreStack and support for other anomaly detection tools/services are in the pipeline.

Azure Anomaly Detector

Configuring Anomaly Detector in Azure

Before you integrate the Azure Anomaly Detector with CoreStack, an Anomaly Detector resource must be created in any of your Azure Subscription. Perform the following steps to create an Anomaly Detector resource.

  1. Login to the Azure portal.
  2. Select Cognitive Services, then click on Create Anomaly Detector resource.
  3. Provide the necessary details: Name, Subscription, Location, Pricing Tier, and Resource Group.
  4. Click Create. An Anomaly Detector resource will be created.
  5. Navigate to the resource page and copy the Anomaly Detector resource's endpoint and any one of the API keys.

Refer Azure documentation for more details.

Integrating Azure Anomaly Detector with CoreStack

Once the Azure Anomaly Detector has been created, the same can be integrated in CoreStack by performing the following steps:

  1. Click Settings icon on the bottom left of CoreStack and select Integrated Tools from the menu. Integrated Tools screen will be displayed.
  2. Select Anomaly Detector in the left side under Monitoring section.
  3. Click Add Account button.
  4. Provide the following details to add the Azure Anomaly Detector.
FieldDescription
Account NameSpecify a unique name for the Azure Anomaly Detector.
DescriptionProvide a detailed description about the Azure Anomaly Detector. It is an optional field.
EnvironmentSelect the type of environment that will be handled by the Azure Anomaly Detector. The options are: Production, Staging, QA, Development and All.
ScopeSelect the required boundary to define the area of influence for the Azure Anomaly Detector: Account, Private and Tenant.
Anomaly Detector EndpointSpecify the endpoint of the Azure Anomaly Detector resource.
Anomaly Detector KeySpecify the API key of the Azure Anomaly Detector resource.
  1. Click Next. The Tools Configuration screen appears.
  2. Select the cloud accounts that must be configured for anomaly detection in the Applicable Cloud Accounts field.
  3. Provide a value for the boundaries of anomaly detection in the Sensitivity field.
  4. Provide a baseline value that must be considered for anomaly detection in the Baseline Setting field.
  5. Click Next. The Authorization screen appears.
  6. Select the required roles to which the integrated Azure Anomaly Detector must be available in the Assign Roles section.
  7. Click Finish.

The Azure Anomaly Detector will be integrated with CoreStack successfully and starts detecting the anomalies in the configured cloud accounts.

Navigation

After an anomaly detector is integrated with CoreStack and configured for anomaly detection in the required cloud accounts, relevant information and insights will be available in the following sections.

Click on Operations in the Left navigation menu and select Anomaly Detector option to land in the Anomaly Detector screen.

There will be 3 tabs available in the Anomaly Detector screen: Metric Anomalies, Activity Insights, Recommendations.

📘

Note: Activity Insights and Recommendations sections are available only for AWS. Support for the other clouds is in the pipeline.

Metric Anomalies

The complete list of anomalies detected for a selected category of resources in the cloud account can be viewed in this section. The analytical details of the anomaly will be displayed as a graph created based on the metrics gathered.

Activity Insights

Activities occurred in the resources that led to the anomalies can be viewed in this section. CoreStack identifies and lists the possible activities that could result in the observed anomalies using its intelligent algorithm. It helps in identifying the critical incidents that does not conform with the normal behaviour of resources.

Recommendations

In this section, recommendations for resolving the anomalies are provided based on the activities detected in the resources that led to these anomalies. It helps you in identifying and performing necessary actions that could potentially resolve these anomalies observed in resources.

Viewing the Detected Anomalies

In the Metric Anomalies tab of Anomaly Detector screen, the list of abnormalities detected can be filtered and viewed for each resource type.

  1. Select the required cloud provider from the Cloud Services dropdown list to filter the anomalies based on cloud.
  2. Select the cloud account for which the anomalies must be viewed from the Cloud Accounts dropdown list to view the anomalies specific to particular cloud accounts.
  3. Select the category of cloud resource from the Category dropdown list to filter the anomalies further based on the resource type. The supported resources are:
CloudCategoryResource Type
AWSInstancesCPUUtilization, NetworkIn, NetworkOut, DiskReadOps, DiskWriteOps
AWSDatabasesCPUUtilization, DatabaseConnections, FreeableMemory, ReadIOPS, WriteIOPS
AWSBucketsBytesUploaded, BucketSizeBytes, BytesDownloaded, NumberOfObjects
AWSTransit GatewaysBytesIn, BytesOut
AzureComputePercentage CPU, Network In, Network Out, Disk Read Operations/Sec, Disk Write Operations/Sec
AzureStorageTransactions, SuccessServerLatency, Egress, Ingress, UsedCapacity
AzureDatabasescpu_percent, dtu_consumption_percent, deadlock, connection_failed, storage_percent
AzureNetworkPacketsInDDoS, BytesInDDoS

The relevant anomalies will be listed and can be filtered using the Daily, Weekly or Monthly filters. Clicking on each anomaly will display the dataset gathered for the resources with the anomaly highlighted in a graph.

Tracking the Anomalies

In the Activity Insights tab of Anomaly Detector screen, the activities associated with the anomaly will be listed providing insights about the sequence of events that have affected at any time a specific function of the resource. Also, activities for only the supported categories will be listed. The supported resources are:

CategoryDescription
Suspicious_IP_AddressThis category specifies whether any event has occurred from multiple IPs within a short span of time.
Security_Group_Rule_ChangesThis category specifies whether any security group rule changes has happened. CoreStack will group all the transactional activities performed around the same time.
Multiple_Termination_AnalysisThis category specifies the multiple termination activities that happened within a short span of time. CoreStack will be group all the transactional activities performed around the same time.
Odd_Time_ActivityThis category specifies whether any event has occurred in an abnormal time (which is not the usual time the user performs the activities).
Multiple_IAM_ChangesThis category specifies whether multiple IAM-related events has been performed within a short span of time.
Multiple_Events_by_NewuserThis category specifies if a new user, who was created in a particular day, has done multiple transactional activities within a short span of time.
Frequent_ActivityThis category specifies if an existing user has performed multiple transactional activities within a short span of time.
Multiple_Login_FailuresThis category specifies if a user has attempted to login through console and failed for multiple times repeatedly within a short span of time.
Threat_Protection_and_Guard_duty_EventsThis category specifies the activities which are the findings of the cloud-native finding tools such as guard duty results, etc.

Resolving Anomalies using Recommendations

In the Recommendations tab of Anomaly Detector screen, recommendations are provided to resolve each anomaly observed. The proposed action for every recommendation varies based on the resource type and the nature of anomaly. You can review the recommended actions for an anomaly and either perform the proposed action or skip it.