Server - ❌FULL OUTAGE System was down

Reported - 08/01/2026 ~16:45

Resolved  - 08/01/2026 - 16:54

Root Cause - The cause was a failing nvme, which has now been physically removed from the server, so it can not interfere with operations again.

Server - ❌FULL OUTAGE System was down

Reported - 06/01/2026 ~16:04

Resolved  - 06/01/2026 - 16:20

Root Cause - The underlying cause was a new nvme that got added which partly failed. (not the same nvme as the one that failed 08/01/2026) . Something we haven't had issues with before in adding new storage. We are writing out new process for adding storage that should be less impactful on the virtual machines should this happen again.

Server - ❌FULL OUTAGE System was down

Reported - 24/06/2025 16:07-18:30

Resolved  - 24/06/2025 EOD

Root Cause - Extremely rare simultaneous failure of both the primary and backup hardware at the data centre.

Server - ❌FULL OUTAGE Users were unable to log in as the system was down.

Reported - 13/11/2024 10:00

Resolved  - 13/11/2024 10:24

Root Cause - A memory leak caused the server to run out of memory. Investigation ongoing to rectify.

Files - ⚠️PARTIAL OUTAGE Users were unable to use the file uploader (users could still drag and drop files).

Reported - 01/07/2024 17:49

Resolved  - 03/07/2024 02:28

Root Cause - developers received notification of a zero day exploit a networking utility running on our file server (OpenSSH). Upon updating, a bug in the new version caused issues for file upload due to the strict security protocols that we have in place.