I'm a professional graphic artist and I have been working full time for almost 10 years now for a software company. Over the years, I have been involved with UI design from a graphic aspect and while doing so also read up on the subject extensively, so, while not being an accredited UI expert, I'm not ignorant of the subject either.
I use several professional apps at work, and use Gimp at home for simpler tasks. The issues I attempted to solve are ones that bugged me at home as a Gimp user and I encountered them mainly by comparing them to programs such as Photoshop and Inkscape at work.
I first have to mention that I made it when I thought a single window interface will never be implemented because of developer resistence to the idea. This is the reason for the separate window.
The problem I was attempting to solve is twofold: UI space saving, and toolbox usability and consistency.
I'm going to posit that it is a good thing to group similar tools; if all the transform tools are together, it is easier for the user to look for one. Same goes for brush-type tools, measuring / helper tools, etc. It is also a good thing if an interface is alway consistent and does not move: a user gets used to icons being ina certain place, and the mouse movement speeds up when they don't have to search for tools but know where they are from experience.
Let's take the right-hand dialog stack (layers, brushes etc) as a given for now. Because of Gimp's rather large controls, you normally don't have enough space in the right-hand stack to add the tool options, so they are in the left-hand stack below the toolbox. So, we have a stretchable window housing the toolbox and the tool options. Since widget size differs among operating systems, you sometimes need to stretch the window so you can see all of the options. This reorders the rows in the toolbox so that tools move about from one row to another, making it impossible to organize similar icons into logical groups, and making a consistent UI impossible.
Another problem with the toolbox as it is now, is that if you use a two-row or one-row solution the tolbox can be well-organized and doesn't take too much space, but you can't attach the tool options to it because it's too narrow, and their geometries makes it difficult and uncomfortable to place them one below the other.
Then again, if you stretch the toolbox so you can attach the tool options, you get a confusing grid of similar looking icons (again with no consistent logical grouping), and you have to spend a few seconds to hunt-and-peck for the right icon when you want to switch tools.
Yet another issue was that the tool options take a lot of space because they are in a single window with stacked elements (sliders, check boxes, etc) which are basically horizontal in nature.So the dialog's width must be the width of the longest element in the dialog - usually a slider. Other elements such as checkboxes have a lot of wasted space to their right. Then there's the fact that most tools have very little options, so most of the dialog is empty space anyway at most times.
So a horizontal toolbar solves the space problem by stacking the tool options horizontally; there is no longer any wasted space. This is nothing new; Adobe apps, paintshop pro and Inkscape all reached the same solution.
You might notice I also replaced the checkboxes in the tool options with image buttons, which present a larger mouse target and are more easily recognized for their function (the user does not have to read the label to find one checkbox out of several similar ones). If you have too many options, just remove the lables and stick with buttons.
As for the toolbox, icons are easier to reach quickly when they are spread out like that and their order and place in the screen is consistent and logical. A single row also presents a larger error tolerance that a grid of icons.
Implementing as a single vertical row on the side is the currently accepted solution in most graphic apps, and is probably the better choice, especially when you can stick the icons to the screen edge.
As the person who drew the horizontal toolbox, I'd still like to present a case for it. It has benefits that probably don't offset the problems, but still it is worth discussing. Here are my reasons for thinking of it at the time:
One reason was that, as I said, I believed the devs would never go for a single-window UI, and therefore the toolbox would be a seperate window. So, combining the two was a must. This reason is now invalidated.
Secondly, I wanted to make it obvious ofr a new user that the tool options were related to the tools. So I added a tab-like visual hint to make it clear which tool was selected and show that the options were related to this tool.
Lastly, Gimp has many tools and I can be sure they'll always fit vertically - practically all screens have more horizontal space. As for the traditional two-row solution, It might make it difficult to separate icons into groups when you have groups with odd number of items.
but again, in a single-window environment it might make more sense to have a single-row vertical toolbox. some tools may have to be eliminated, combined or demoted to dialogs though (I have some ideas aobut that).