Pointwise ergodic theorems for arithmetic sets