Alert Events

Alert Events

Alert events represent the output of the alert processing system as much as ingested events represent its input. Every alert event is the most recent event that caused an alert to be raised. Additionally, it includes a nested "alert" structure that provides information describing why the alert was raised. This "shadowed" structure allows switching in GUI between dashboards showing the original events and dashboards showing the alerts while preserving the same event filters. 

The nested alert structure provides alert-raising context. It explains why an alert was raised, what the events were, and what alerting criteria led to raising an alert. An example is shown below and it contains the following elements:

• alertId: unique ID of an alert as created during calls to API. (setalertconfig)

• desc: human-readable name of the alert

• docid: document id for the alert as stored in alert index in Elastic-Search

• evilSCore: score determined during alert processing.

• gte: greater than, beginning of a time range in which an alert was spotted. Used to narrow-down the events which caused the alert.

• ipblacklistsing, ipBlacklistingReason, ipwhitelisting, ipWhitelistReason: if blacklisting/whitelisting is turned on and the event includes an IP address, information about it is stored here.

• key: description of the key for which the alert was raised

• keyScore: sum of the score for multiple  alerts that referred to the same key; 

• lte: less than, end of a time-range in which an alert was spotted. used to narrow-down the events which caused the alert.

• reason: additional alert-type specific information that describes the alert-raising conditions. here, the "custom-key ratio" alert found that the ratio 4/12 exceeded the 30% threshold.

• score: summary of how the alerts added up to the alerting score.

• severity: alert severity as configured in the alert's configuration.

• tenantId: tenant ID

• test: alert type (here: CKR stands for custom-key ratio)

• throttle: indicates if the alert was throttled. If set to true, the alert was not raised because the same alert for the same key was raised recently.

• throttledTillTS: timestamp indicating till when an alert remains in a throttled state for the given key.

• throttlePeriod: length of a throttling period, taken from alert configuration.

• timestamp: when the alert was generated

• uriBlacklistReason, uriWhitelistReason ... like for IP

Additional elements are included for automated processing and debugging. Names of such are prefixed with underscore as a hint to GUI that the extensive information may be concealed for brevity, values are stringified to keep Elastic's number of attributes contained.

• elastic filter is an Elastic Search expression that narrows down the ingested events to those that caused the alert to be raised. They include time range, key filters and configured filters. "_elasticFilterS": "{\"elastic\":[{\"range\":{\"@timestamp\":{\"gte\":1727252523000,\"lte\":1727253008000,\"format\":\"epoch_millis\"}}},{\"query_string\":{\"query\":\"attrs.type:/call-attempt|call-start/\"}},{\"match\":{\"attrs.dst_ca_name\":\"proxy\"}}],\"gui\":{\"lucene\":[\"attrs.type:/call-attempt|call-start/\",\"attrs.dst_ca_name: \\\"proxy\\\"\"],\"timerange_gte\":1727252523000,\"timerange_lte\":1727253008000}}",

• the key's profile snapshot dumps a profile at the moment when alert was raised. This helps during debugging to find out the context at the very moment when an alert was raised. This context changes with every next event processed, and it is thus important to capture the snapshot. _profileSnapshot": "{\n  \"CKR_3W34jB73\" : {\n    \"alertKeyId\" :  \"cstm#attrs.dst_ca_name_-347779663\" ....