Captain Cook had his doubts whether this would give him access inland, and he was right: this is not the place to choose as an entry-point to New Zealand. Even these days you have to make land, cross over the hazardous Wilmot Pass, make another boat trip on Lake Manapouri and then take a bus trip to Te Anau before you can get a good cup of tea. Doing the tourist thing, of course, you do it twice: start at Te Anau, go all the way to Doubtful Sound and then come back again – and it all takes 10 hours and is pretty tiring.
Clearly I did this on a rotten day weatherwise - three boat trips in cold wind and rain - but yes it was certainly worth it. But if you want a gentler experience go for Milford Sound by bus from Te Anau - or even better, the fantastic little aeroplane flight from Queenstown.