Tools for Ensemble Interfaces
TCP/IP Interface Monitoring across all selected Ensemble Namespaces
Over the years, we have found Interface Engine Monitors to be of limited use for TCP/IP persistent connection interfaces because they do not show the socket connection status (CLOSE_WAIT, CLOSED, ESTABLISHED, FIN_WAIT_1, FIN_WAIT_2, LAST_ACK, LISTEN, SYN_RECEIVED, SYN_SEND, and TIME_WAIT). Our TCP/IP Interface Monitor for Ensemble shows these exact statuses so there is no guessing what is going on for the Ensemble side of a TCP/IP persistent connection interface, and we do this with all these flexible features:
* Dashboard/no-scrolling-necessary display of interfaces running in any or all namespaces
* Administrator selected namespaces for monitoring interfaces
* Variable time ranges for when to monitor each interface
* Variable allowed down times for each interface before a phone call needs to be made
* Variable allowed queue dwell times for each interface before a phone call needs to be made
* Variable allowed suspended messages in a row for each interface before disabling it and a phone call needs to be made
* Contact information from the On Call application specific to each interface that is available from the interface monitor with a click
* Email/paging addresses for each interface to send a notification whenever an interface has been down for more than 5 minutes
* Settings for file systems or disk names to monitor free disk space
* Settings for free-disk-space threshold to send an email/page to the contact email/paging low-disk-space addresses
* Email/paging addresses for when the interface monitor isn't running (normally only set for the operations/command center user id)
* Email/paging for all Event Log Errors that do not have user selectable ignore criteria in the text of the error.
* Dynamic down time announcements to alert staff of scheduled outages (only displayed at user specified dates and times)
Link to FREE software: Google Drive (January 2020)
November 7, 2013
The Interface Monitor has a new look. Only the down interfaces have the names shown in the display box and the up interfaces have mouseovers. This allows a large number of interfaces to fit on the screen.
The TCP/IP Interface Monitor can simultaneously monitor productions running in administrator-selected Ensemble namespaces. Monitor productions running in all namespaces or only the namespaces you choose. The Add/Remove list of existing namespaces only appears for administrator users, so only administrators can change which namespaces are being monitored (operations users can't tamper with the settings).
NOTE: This monitor reports the interface status for each interface in a running production that has a class name containing "TCPService", "TCPOperation", "FTPService", "FTPOperation", "FileService",' or "FileOperation", so classes need to be named accordingly to appear in this monitor (this can be customized for registered users). When first run, the monitor will default to only monitor the current namespace, so no additional setup is needed for those users running a production in only one namespace.
Example showing an administrator monitoring 3 namespaces. When an interface is clicked on, the namespace for the interface is displayed in the detail section (lower left).
Example showing an administrator running the monitor with the selections of existing namespaces to add or remove for monitoring. Also, the log display is selected for an interface - namespace for the interface is in the title of the log as well as the detail section.
Example when no production is running one or more of the monitored namespaces (Special Interface Name of ProductionNotRunning appears and gives contact details when clicked on - click on image below to clearly view this functionality):
Example showing a non-administrator running the monitor.
If no interface is selected the free disk space of selected disks (on settings page) are displayed and monitored according to the user selected alert point. If the free space falls below the alert point, an email/page is sent to the addresses entered on the settings page. The free disk space is updated from the system every 5th refresh cycle (approximately 2.5 minutes) so you may need to click 5 times to see your new disk space results if you just made a settings change.
Announcements display during the dates and times selected on the Announcements page:
Monitor showing a disabled interface for exceeding the allowed number of suspended messages:
TCP/IP Monitor Settings screen with the option to monitor non-Ensemble TCP/IP connections.
* Fill in (new) or select (existing) port number that will be used to receive updates from each old interface box (one port number per old box/IP Address).
* Edit "SendNetstat-29000.pl" (on the Intersystems ftp site) to put in that port number and the Ensemble IP address.
* Add the perl program to a directory on the old system and add it to a cron job for UNIX or scheduler for Windows.
* This Perl program sends netstat updates from the old system to Ensemble once every 20 seconds.
All Ensemble interfaces in all namespaces are detected automatically and entered with default settings. Edit interface settings for characteristics that fit the interface's normal behavior. If, for example, an interface often goes down for a few minutes and then resolves itself, adjust the allowed down time from one minute to 3 minutes to avoid false alerts. These settings can be tweaked so that there will never be a false positive alert and operations staff should always make a phone call when an interface goes red.
For disk space monitoring in UNIX, put in a partial or complete name of the file system for those files systems to be monitored (if you add "/dev" to be monitored, then all file systems will be monitored). For monitoring disk space when Ensemble is running in Microsoft Windows, use "C:", "D:", etc. for disks you wish to monitor (every drive you wish to monitor must be added individually).
For the operations user, you can set an email/paging address if the monitor doesn't refresh for 5 minutes (in cases where someone navigated away from the page in the Web Browser or logged out and forgot to return to the Interface Monitor page, or an automatic Windows update to the PC with a reboot, or a network problem with the monitor PC). This monitoring checks that refreshing is occurring by user ID, so only one instance of the monitor should be running at a time for the operations user ID in the hospital command center. If a TPC/IP Interface Montor is running elsewhere under that user's id, refresh failure of the Operations Monitor will not be detected.
Additionally, email/paging can be set for Ensemble Event Log Errors with filtering to block errors that should not be emailed or paged.