From a high perspective, every event is ingested on the system's input. Then it is evaluated using configured alerting rules and aggregated history of previous alert processing, so-called profiles. The ingested events are mapped into streams by the key value: an attribute selected in the event. Each stream is tested separately against alerting criteria. If alerting conditions are detected, an alert is raised for the key using the RESTful interface. Below is a list of steps that are performed for every received event. As an example, think of an alert that is to be raised when a user identified by his IP address fails to authenticate three times in the past five minutes:
INGESTION: A new event is ingested for a tenant. Depending on the ingestion method, the initial data is transformed to make it processable. The transformation method depends on the specific source of data. For example, the original data may be uploaded using SFTP or HTTPS, encoded as JSON or comma-separated values. At the end of the ingestion procedure, one or more JSON events are generated for further processing and assigned to a tenant.
CONFIGURATION: The tenant’s configuration is read (configuration is stored in Elastic, cached on filesystem and accessed using API) (readTenantConfig). The configuration contains three parts: alert configuration, transformation script, and variables. The most important part is the alert configuration: it defines the types of alerts and their parametrization. In our example, the configuration specifies a rate alert set to fire upon the third authentication failure within five minutes. The configuration includes two more optional parts, neither used in our example. A transformation script is an optional custom Javascript function that can be applied to the ingested event. It is used for custom transformation, such as conversion of units in events or normalizing telephone numbers to E.164. The third part, variables, may further parameterize the transformation function.
TRANSFORMATION. Optionally, the transformation code is applied to the incoming event (transform).
CONTEXT CREATION. Alerts are chosen for processing only if enabled, and the incoming event includes all attributes referred to from configured keys and filters. (tests2Contexts).
KEY: Alert’s key (for example, IP address, URI address, trunk ID) is determined from the ingested event, say 10.0.0.10. This is repeated for every alert in the tenant's configuration (gatherAllKeys).
PROFILE READ: The keys' histories, AKA “profiles,” are read for every key found (loadProfiles). (in our example, the profile may state that 10.0.0.10 failed to authenticate two times in the past minute)
COMPUTE: The alert criteria are computed for every alert active in the current alert configuration. In our example, the current event's occurrence is added to the previous two stored on the profile, which causes the alert's threshold of three to be exceeded. The alerting score is increased for the affected key by a value specific to that alert type, most often 100. (updateTotals)
MANAGE: Additional criteria are considered before an alert is raised: (filterRaisableAlerts)
To be eligible, the sum of all alert scores referring to the given key must exceed a threshold. (Neither a single alert's score nor the sum of all alert scores.) The threshold is set in the environment variable scoreToAlert, defaults to 40.
throttling (Was the alert recently raised for this key?)
suppression (was the alert suppressed by the user via API for this key?).
PROFILE WRITE: The 10.0.0.10 profile is updated to include information about the most recent authentication failure and raised alert (createUpdatePromises).
ALERT PROPAGATION: If eligible, an alert is raised via restful (generateExternal) and archived in Elastic. Only the first generated alert is sent out if multiple threads attempt to raise the same alert concurrently.