Star clusters and binaries in the first galaxies