Here is the step-by-step process I used to Jailbreak my iPhone 4S. I used the software redsn0w and definitely recommend it. I've been running it for weeks and have had no crashes.
Cydia is a Jailbroken-only application. It's the App Store for Jailbroken iOS devices, used to download Applications and Sources.
1.) If you get any errors when trying to update Cydia applications, try this: Cydia Error Fix's