The acronym ROCKSTAR stands for Robust Overdensity Calculation using K-Space Topologically Adaptive Refinement (cool, huh?). The ROCKSTAR algorithm groups particles together, similarly to FoF, but it also takes into account the speed and direction of the particles in the halo as well as their evolution in time.

This results in much more physical halos with less arbitrary shapes (i.e. more spherical) compared with FoF. This algorithm isn't as widely used yet compared with FoF because it takes much more processing power and requires more memory during runtime.