After I bought a Korg Poly-800, and an upgrade kit for it, I also started looking at the circuit bending scene.
using PIC microcontrollers and Basic Stamps
I also found an old box with an experimental electronics lab for kids starting at age 10.
Here's a video of me playing with a homebrew oscillator:
The box includes a book with all basics explained in 200 examples. There must be a shorter way to learn these basics ...
How Stuff Works
Electronics tutorials: Oscillator basics
Circuit Benders
Software
Kicad : create electronic schematic diagrams and printed circuit board artwork.
Hardware
Arduino : an open-source electronics prototyping platform.
Conrad.be : an online electronics store. (moer, knop, potmeters, desoldeertechniek ...) tripleaudio.nl is another one.