A stable rocket requires that

the Center of Pressure (where the aerodynamic forces balance; #2 in the diagram) 

is behind

the Center of Mass (where the mass balances; #3 in the diagram)

We're going to play fast and loose with physics terms here. For our purposes:

Fins move the Center of Pressure backward

More precisely, fins add relatively more surface area than mass, so they pull the Center of Pressure more than the Center of Mass. If you put the fins at the back, it pulls the Center of Pressure rearward more than the Center of Mass => more stability

Payloads move the Center of Mass forward

More precisely, payloads add relatively more mass than surface area, so they pull the Center of Mass more than the Center of Pressure. If you put a payload at the front, it pulls the Center of Mass forward more than the Center of Pressure => more stability

Play with Fins and Payloads until the rocket is stable

When stable, it will point the direction of travel, into the wind. Like a windvane. 

Now you know more than I did for my first two years of this project while I was preoccupied with getting the launcher to work. Let's discuss Fins and Payloads in the next sections.