There is some software built on top of TeX with which one can type set music. This software is available through 4 different packages: MusixTeX, MusiXLyr, PMX and M-Tx. The latter two are frontends to the MusixTeX and have a much simpler language; PMX handles just instruments whereas M-Tx builds on PMX and allows lyrics (using MusiXLyr) as well. They are all available from here and represent the substantial open source efforts of many people.
For a long time I used the TeX-based software. Recently (2006), I explored some other systems as I was not happy with some aspects of the Tex-based stuff.
After looking at a few alternatives, I haved settled on Philip's Music Writer (PMW), which you can find here. PMW does not try to be quite so clever in automatically making adjustments for you, and consequently is more predictable. It still has lots of flexibility, and its language is well thought out, consistent and easy to use. It is also very well documented. Its author, Philip Hazel supports it with no need for recourse to community mailing lists.
The other main package I looked at was LilyPond and used it to typeset one piece. However, I found its interface fabulously confusing and bulky, and in particular the MIDI support was not very good.
I have type set some of my compositional efforts using these various software packages. I provide the output as PDF and MIDI (rather rudimentary; no dynamics, no ornaments, no touch marks, no arpeggiated chords, vocal lines are given to a synthetic voice and so on).