A cluster is a parallel computer build from commodity components. In the 2000's, these clusters were build with the performance available in commodity personal computers. Over the next two decades, these high performance computer clusters have moved to much more sophisticated and capable server units. The purpose of PROJECT KOIOS is really a throwback of using the same technologies, but on low voltage computers.
In a sense, this cluster is a throwback to older approaches of cluster computing using what is truely commodity hardware rather than high-end off-the-shelf computational nodes. However, we should temper our project expectations. For the most part, I am interested in using KOIOS as a development platform. I will add nodes as necessary to meet the kind of workflow tasks I am interested in.
Part of my motivation for this server is the difficulty of doing parallel programming development in an HPC production environment, which I worked on at Lawrence Berkeley Labs, Idaho National Labs, the University of Florida, and the Ohio State University. There is high contention for nodes and I have found that queue management to be somewhat difficult considering that there maybe a lag measured in minutes and sometimes hours, which is completely unsatisfactory for ad hoc development. There problems are even more problematic when you are trying to automate the setup, submission, and analysis of simulations.
Hardware
At the very least, I expect this computational cluster to meet my development requirements. PROJECT KOIOS at the current time is interested in setting up cluster computing for three types of computing platforms: (1) the Intel NUC, (2) the Raspberry Pi, and the (3) NVIDIA jetson nano.
Operating System
I am looking to integrate the NVIDIA Jetson Nano into my project. The NVIDIA project boards uses NVIDIA JetPack which is a derivative of Ubuntu 18 LTS. I am using Ubuntu 20 LTS throughout the project. Ubuntu is the most popular Linux distribution and should have good support if I extend PROJECT KOIOS to the cloud.
However, most high performance computing (HPC) clusters use CentOS which is derived from Red Hat Linux. Ubuntu is derived from Debian and has small but significant differences in the file structure (which I don't think will matter).
Credential management
When I was first a UNIX administrator, I used NIS/YP to manage login credentials between all the various computers. Considering that KOIOS is starting with 12 nodes and may expand, one of the first projects that we will establish is login credential management.
NIS is fairly easy to setup, but has almost no security
LDAP
FreeIPA (identity/policy/auditing)