The alert processing system produces logs that have the following structure:
<timestamp> <context> <filename> <text>
Context identifies which processed request a log line belongs to. This is important as in the nodejs asynchronous model, processing, and logging of multiple received events are often intervened. Context takes one of the following values:
Timer-driven tasks show a well-known description of their purpose, like "periodic-redishealth" in the example bellow. Other values include "periodicConfig" (background task that refreshes cached configuration for use by ingestion plugins) or names if polling-based ingestion plugins.
Logs for ingested events show the received 'x-debug-id' header if present ... (useful to correlate auto-tests with logs)
... an ephemeral event-by-event uuid.v1() otherwise.
2024-05-10T23:18:48.463Z periodic-redishealth alerts/mkalerts.js DEBUG: alert CSM:NFwgCbJ throttled because elapsed 420 sec < 600 threshold
2024-05-11T05:38:34.446Z b542b040-0f58-11ef-a6ed-8332a95f47ad alerts/sipTracker.js TTT onSipSessionEnd cancelTimer trying for ID NKTqMxy+kREiNyRJMMM7kQ-83754896@192.168.1.87-1946023864 key {"type":"tenant","name":"tenant","value":"5340085e-0e10-4b3c-8f4b-2fefe517bf6b","id":"tenant_-1644031141","origval":"5340085e-0e10-4b3c-8f4b-2fefe517bf6b","encrypt":"plain","orig_encrypt":"plain","profile":"tenantprofile","attrName":"tls-cn","keyRef":"eyJuYW1lIjoidGVuYW50IiwidmFsdWUiOiI1MzQwMDg1ZS0wZTEwLTRiM2MtOGY0Yi0yZmVmZTUxN2JmNmIiLCJlbmNyeXB0IjoicGxhaW4ifQ=="}
The logging level can be increased globally to debugging by setting the environment variable "debugFlag" to "true."
The level can be adjusted on a per-source basis using the API call "mlogging". It takes identification of the source file (directory/filename) as one parameter and desired level as the other. The level can be "debug", "info", or "warn"; "warn" is enabled by default. '*' can be specified as source file paramater, then the change of logging level applies to all files.
Alternatively, one can use the "slogging" API and matching expressions to identify incoming events whose processing will be logged at the "debug" level.
The API-controlled changes remain in force until the next nodejs reboot.
curl --insecure 'https://localhost:4000/api/alertapi/mlogging?module=alerts/sipTracker.js&level=debug'.
curl --insecure 'https://localhost:4000/api/alertapi/slogging' -d '{"expr":"attrs.type~\"ingest_redishealth|call-start|call-end\""}'