Ingestion plugins support the collection of data from multiple sources. The incoming data is ingested using various protocols (HTTPS, SFTP for example) and data formats (often JSON, sometimes CSV) and transformed into JSON by the plugins. There are two kinds of plugins: asynchronous and polling plugins. The synchronous mode plugins listen for incoming events at a well-known path (e.g.,/ingestion/zoom/4a6b89a4-0b76-11ee-be56-0242ac120002); some must initially subscribe to the source to receive the incoming events. Polling plugins attempt to retrieve events periodically. Both plugin types are configured using special configuration variables. The configuration information typically includes some authentication information. Often, the sources must be configured on their side as well.
A logstash-facing module handles events ingested by logstash from frafos SBC or frafos SIP probes.
mode: asynchronous (ingestion path: /ingestion/logstash/<TENANTID>)
configuration: -
status: mature
A module that keeps pinging a SIP service in the background and generates events reporting on health/latency of the service. The "pings" are sent every minute, a timeout occurs within two seconds.
mode: polling
configuration:
To-URI (required) ... resolved using DNS SRV
outbound proxy host:port (optional)
request-URI (optional)
transport: tcp|udp (optional)
status: experimental
limitations:
only TCP and UDP (no TLS)
no digest authentication
no UDP retransmission
Example event:
{
"@timestamp": "2024-11-28T09:38:28.165Z",
"tls-cn": "7716afd1-2b57-4c78-802d-1400a222f65b",
"type": "ingest_sipping",
"attrs": {
"from": "sip:openrcs.com",
"to": "sip:openrcs.com",
"method": "OPTIONS",
"r-uri": "sip:openrcs.com",
"transport": "udp",
"source": "10.89.0.2",
"sip-code": 200,
"sip-reason": "Okey",
"src-port": 34987,
"call-id": "859b85d0-ad6c-11ef-8973-ebd06d53d970",
"elapsed": 19.274578005075455,
"type": "sipping",
"errors": []
}
}
A module that keeps requesting a JSON document from an HTTP(s) service in the background and generates events containing the JSON answer and latency. The requests are sent every minute, a timeout occurs within four seconds.
mode: polling
configuration:
URI (required)
header: a JSON object representing HTTP headers (like in {"x-header-name": "value"}) (optional)
basic authentication password (optional)
status: experimental
limitations:
only TCP and UDP (no TLS)
no digest authentication
no UDP retransmission
Example event:
{
"@timestamp": "2024-11-28T11:03:12.366Z",
"tls-cn": "7716afd1-2b57-4c78-802d-1400a222f65b",
"type": "ingest_jsonget",
"attrs": {
"elapsed": 148.8181409998797,
"url": "http://headers.jsontest.com/",
"answer": {
"sec-fetch-mode": "cors",
"X-Cloud-Trace-Context": "93075842ddf488c1afcbd69c18bbcbc9/10128218636280095718",
"Authorization": "Basic aGo=",
"accept-language": "*",
"foo": "bar",
"traceparent": "00-93075842ddf488c1afcbd69c18bbcbc9-8c8ea93026f763e6-00",
"host": "headers.jsontest.com",
"accept": "*/*",
"user-agent": "node"
},
"type": "ingest_jsonget"
}
}
a plugin for internal use -- it periodically retrieves the status of backing REDIS memory and passes it in the form of an event to alert processing under the system owner's ID. The events are of type "ingest_redishealth".
mode: polling
configuration: -
status: stable
current weather data can be downloaded from the weatherlink cloud.
mode: polling
configuration: weatherlink key and weatherlink secret
status: stable
current measurements from a heatpump can be downloaded from daikin cloud
mode: polling
configuration: daikin user name and password
status: outdated (requires catching up with a change in the API)
a plugin to subscribe to CUCM CDR webhook
mode: asynchronous (via SFTP)
configuration: -
status: outdated, needs an udpate
a plugin to subscribe to zoom webhooks
mode: asynchronous
configuration: zoom secret
status: outdated, needs an update
a plugin to subcsribe to MS Teams webhooks
status: outdated, needs an update
mode: asynchronous
configuration: Microsoft user, secret, tenant ID, resource identification
limitations: varying webhook latency, not real-time
a plugin to subscribe to Cisco Cube CDR webhook
status: outdated, needs an update
configuration: Cube IP address
mode: asynchronous (ingestion path: /i/)
limitations: only CDRs (call-starts come same time as call-ends), insecure: it uses HTTP (no HTTPS), it has no message integrity check. it also can't deal with long URIs therefore to deal with it, the ingestion path uses /i/ path in departure from the htts://HOST/ingestion/ciscocube/... schema and UUID is not assumed to be in the path (it is conveyed in payload).
a plugin to subscribe to webex call hooks
mode: asynchronous
configuration: webex secret, refresh token, access token
status: outdated, needs an udpate