Embedded Files
Posted by Uncle Bob on 11/21/2009
There has been an increase in interest in Software Katas of late. What is all this buzz about, and why might it be important?
Several years ago (Pragmatic) Dave Thomas started a special blog about coding katas. The idea was simple: Professionals practice.
This thought might not have occurred to you before; but it’s self-evidently true. Professional musicians practice. Professional dancers practice. Doctors practice. Soldiers practice. Indeed, any kind of professional craftsman or artisan must practice their adopted trade in order to execute it well when it counts.
Dave’s point was that professional programmers need to practice like any other professional. He set forth a number of simple exercises and problems that programmers could solve during their practice time. He also suggested that practice time needs to be scheduled and protected as part of the normal routine of a software developer.
At XP2005 in Shefield, UK, I attended Laurent Bossavit’s and Emmanuel Gaillot’s Coding Dojo session. I blogged about it at the time. I left that session with a different view of what a Kata might be. In Dave Thomas’ view a kata is an exercise to solve. Each time you solve it you try a different approach. Dave’s notion is that a kata is a way to explore the solution space of a problem. But in Martial arts a kata is something different; it is a set of motions that one memorizes in minute detail. A martial arts kata is a precisely executed dance. I left Sheffield with the idea that a coding kata could be more like a martial arts kata.
About a year ago, Corey Haines (our wandering minstrel of code) stopped by and told me about some mini-conferences that had adopted the notion of coding kata. What he described sounded more like a group dance than a bunch of people independently solving a given problem. This got me thinking. So at the next Software Craftsmanship meeting at 8th Light (Which, not coincidentally, takes it’s name from a martial arts philosophy) I conducted a simple kata session in dojo style.
I, acting as the Sen Sei, had everyone file in to the room as they would to a dojo. I had them put their green bands on their wrists (mine was black!). I bade them to sit, and then gestured for silence. For the next 15 minutes I led them through the Prime Factors Kata in Java without uttering a word (well, almost). I would write a simple test case, then gesture for them to do the same. Then I would make the test pass, and again gesture for them to repeat my moves.
Then, this Fall, I attended Programming with the Stars at Agile 2009. As I watched the contestants perform their programming sessions, I realized that they were not simply demonstrating programming prowess; indeed they were competing in how well they performed their session. That’s when it struck me. This was a performance art!
It seems absurd to think of programming as a performance art. And yet watching those people up on stage working through routines that they had clearly practiced over and over again, was delightfully entertaining to me, and to the crowd. There were cheers and applause when a performing pair did something especially well. There was laughter when a pair executed a clever joke in code, or in behavior. For programmers, watching other programmers perform well practiced routines was fun!
Now I had seen martial arts katas executed as performance art during competitions. Indeed, it is a beautiful thing to watch a skilled and well practiced martial artist peform a kata. And that started me thinking. Why would a martial artist practice these forms so intensely that they could be performed as an art?
The goal of martial arts is not to perform on stage. The goal of martial arts is self defense! And yet, when a martial artist is at the peak of his skill, his practice takes on the quality of a performance art. When you watch a skilled martial artist perform, you know you don’t want to fight him. The performance is a demonstration, and a realization, of mastery.
And yet the performance is not the goal. No martial artist practices his art so that they can perform on stage. A martial artist practices to achieve personal perfection in the art of self defense. The fact that the practice can be performed is a (pleasant) side effect.
Could it be that these coding kata that we’ve been dabbling with for the last few years could be performed? Is there a benefit to practicing them so much that performance is an option? Is there really any reason for a programmer to create and then memorize a routine down to the level of individual keystrokes? Is there profit to be gained from practicing that routine so thoroughly that you can do it at high speed with virtually no errors? This August I decided to find out.
I started with the Prime Factors Kata in Java, and set it to music inspired by this video of dancing kites. The piece is about 4 minutes long, and I figured I’d need about twice that, so I found two different version of the music and played them back to back. But working in Java is ponderous, and I could not complete the kata before the two pieces had finished. So I switched to Ruby. Using Ruby and Rspec I was able to finish the Kata with plenty of time to spare.
Thereupon began an evolution. Since I had time, I added more features to the kata. As I improved in my practice I found I had even more time, so I added even more features. I refined and polished. I refactored my refactorings. I trimmed keystrokes, and learned many new keyboard shortcuts in the process. I completely abandoned the mouse; and then later reacquired it for flourishes and highlights (but never for actual coding).
Week after week, I set aside 30 minutes or so every day to practice, and practice, and practice—just the way I used to practice my Jiu Jitsu Katas. I would practice on airplanes while flying to clients. I would practice in bed with my laptop in my lap. I would practice late at night in hotel rooms after a long day consulting for clients. I would practice early in the morning before shower and breakfast. And the practice started to pay off.
Ten weeks later I finally decided I was ready to create a screencast. I reduced my screen resolution to 1024X768. I configured SnapZ ProX to record the whole screen at 80% size. And I began to record.
Performing a coding kata in time to music is a very difficult thing to do. Maintaining the timeline is critical. There are very few errors that you can effectively recover from. If you miss a stroke, the whole sequence melts down. And melt down it did. Over and over and over again. I spent weeks trying to record a reasonably good session. I must have done hundreds of takes. It was very frustrating.
But day by day I got better and better at it. Finally, after weeks of trying, I got what I consider to be a recording that, though not perfect, is good enough to present. You can watch that recording here.
Has this paid off in the regular programming I do? I think it has. My mouse usage is much less than it used to be. I know a much larger number of keyboard shortcuts than I used to, and I can use them almost instinctively. My typing and accuracy have improved quite a bit, and I now type more punctuation and number keys without looking. So, yes, I think the effort has paid off; though I’m not at all sure the payoff compensates for the effort.
But something else compensates for the effort. Making this recording was fun—it was a lot of fun. And that’s probably the real answer to the question posed in the title of this blog.
Comments
Jon B about 3 hours later:
Interesting video. I’d have never thought to be in a situation where critiquing a programming piece based on it’s use of musical phrasing is a possibility. If nothing else I think it lends credence to calling coding a craft not a science.
While i was reading some of the text on katacasts I had an odd thought (may have been echoed by others so no claim to being the first to think this). One thing a friend mentioned about some of the katas he did (i think in kung fu but quite possibly wrong) was that he was paired with another student who was doing the “opposing kata”. That is to say when one was doing a strike, the pair was doing the appropriate block.
It might be an interesting experiment to attempt a dojo style kata with pair programming built in. The hard part would be making it useful without being hokey.
Kind of off topic, at around 6:56 you move the initialisers for factors and divisor onto the same line. I would have thought that having them on separate lines was cleaner. Is preferring them on a single line a rubyism, or are my old c defenses getting in the way?
Frank Quednau about 3 hours later:
Jon, I thought craft and science are not mutually exclusive. Historically, I’d say craft usually comes before Science.
David Vrensk about 4 hours later:
Watching the kata was a great way to start the day (I caught your tweet before I got up, in UTC+1). I think it’s interesting that programming katas are returning to the roots, so to speak, and I think it is time that we had different words for when we try to solve a known problem in our own way and when we try to repeat the ways of a master. Both are useful ways to practice programming and deserve their own fora.
Since this is a kata of the copy-the-sensei kind, it would be really useful to know what keystrokes you were using. E.g., I believe that Corey Haines uses KeyCastr in his LCD Numbers Kata.
BTW, the link to Textile syntax above the comment box is broken since the sad disappearance of Why the lucky stiff.
Johannes Brodwall about 4 hours later:
Jon, I’ve performed a longer kata with a partner. Both we and the audience seemed to like it.
An interesting point was that with this hour long kata, the shortest it’s taken us while pairing has been 20% less than the shortest it’s taken solo. I was surprised to see a time benefit from pairing after having practiced a lot solo.
Bob, enjoyed your video. Thanks for showing us.
Jon Kern about 4 hours later:
En lieu of holding up a lighter to get an encore, kudos :-)
Perhaps solving the Navier-Stokes existence problem next?
(Now to go out and find a fixed-price project that needs a Prime Number component in ruby!)
Nobody about 10 hours later:
I just watched your video. Nice :-).
I am no Ruby coder so I can not really judge if it is good Ruby style to do more than one thing per line. (Given that Ruby has some roots common with Perl that might be the case…)
Regarding your Rspec: You are treating the whole functionality as a black box and are only asserting if several results are correct. But that way IMHO you are not really making any requirements explicit. To me the biggest benefit of a BDD style is that in the end you end up with executable requirements/specs and not just examples that seem to work. So I would suggest having something like the following as a spec: 1. primeFactors(n<1) => undefined 2. primeFactors(n=1) => [] 3. all elements of primeFactors(n>1) multiplied == n 4. every element in primeFactors(n>1) is a prime number
Luca Minudel 1 day later:
I find useful the training with Kata to myself to improve how I work with limited time (setting priorities) and how I recover from errors (dealing with unexpected events). Since I’m primary a .NET/VS guy I can also simulate uncertainty working with Java/Eclipse For this scope I find interesting to try always different katas.
Maybe now it is time to give a try also with the training you suggested.
Eric 1 day later:
Bravo for the performance!
Jeff Anderson 1 day later:
awesome,
that must have been murder to get right…
dfo gold 1 day later:
I find useful the training with Kata to myself
Gøran Hansen 2 days later:
two words: amazing performance!
Thank you! You Prime Factors kata video inspired me.
Michael 5 days later:
One word: Thanks!
frasi 5 days later:
Been a programmer (in one form or another) since days with TI/40A and Apple IIe. But the Prag folks and, now, your blog is helping to revive the joy in this craft. Thanks!
Vladimir 5 days later:
software development is a bit more than typing exercise.
Andrew Dalke 6 days later:
I worked on your prime number kata a few weeks ago but couldn’t figure out any way to leave comments on that page, so it seems like an opportune time here.
I coded it up myself before looking at your solution. Mine was a dozen or two lines of Python, so I was looking forward to seeing some elegant 2- or 3- lines solution in Java. But when I looked at it, I didn’t even think it worked.
At first I thought it would report 4 and 6 and other numbers as possible factors, since it was testing those values. I looked at the tests, figuring the tests would show me that it did indeed skip those values, only to realize none of the numbers tested had a factor greater than 3. I really would have preferred some tests which contained, say, 5 or better 19 as factors.
That’s not saying your solution is wrong, only that the tests aren’t telling me enough to make me happy that the code is right, without actually looking at the code.
Once I figured out why the code worked and wouldn’t report ‘4’ and other non-primes as factors, my next thought was on how slow the algorithm was. It really is testing every single possible number less than the current candidate, including composites which by construction cannot be factors.
That’s pointed out a difference in how we understood the problem. When I’ve had to deal with primes it’s been for number in the range up to 2 * * 32. I coded my solution for that expectation, and with the ability to scale up a bit higher. I used the Sieve of Eratosthenes to generate the prime factors I used as tests.
Your code doesn’t even take into consideration the observation that you only need to test up to sqrt(candidate) so your performance is O(n) in the worst case, when it should be O(sqrt(n)). By comparison, if you only test prime factors then worst case is O(?(sqrt(n)) or about O(sqrt(n)/ln(n)).
For a test case, try 2 * * 31-1 which is max 32-bit signed int and a Mersenne prime. It will take your code a very long time to solve, while my Python code takes 4.5 seconds, most of which is spent in generating the 6543 primes from 2 to 2 * * 16.
Thinking about your kata, the problem is that it doesn’t provide any sort of idea about domain applicability. If the factors really are only going to be 2 and 3 then it’s easy enough to just use those as the only test factors. In fact, you should be able to pass your tests with “while(;n%2==0;n/=2){primes.add(2);}” “while(;n%3==0;n/=3){primes.add(3);}”.
If it’s a small range, then pre-coded primes would also be a better solution than what you did. There are only 6543 primes to handle all numbers up to 2 * * 31.
I feel like you haven’t developed enough tests to show that your code really works. You need to test numbers which don’t have 2 and 3 as factors. 49(=7 * 7) and 91(=7 * 13) would be good ones.
You also have to have something which tests the range of applicability, like 2 * * 31-1 if you expect numbers that large.
I ended up being disappointed with this kata because it makes no mention of these problems, and uses a solution worse than one understood by the ancient Greeks, as well as many modern high school students. There was no real world use case driving the problem domain and so what is really a toy solution does work, but as it’s a toy solution and underspecified problem, I don’t think it’s one which should be promoted as a good example of a kata.
Even if you leave it as a beginning practice piece, you need to have some commentary to point out the limitations.
Andrew Dalke 6 days later:
Just went through your video at http://www.vimeo.com/2499161 . You brought up the Sieve of Eratosthenes approach but called this “much more elegant”. The only reason it’s elegant is the driving problem is answers for your son’s homework problem. In that case, no prime divisor is going to be larger than perhaps 19 so you can hard-code the first few primes and use those as candidates. The result will be faster than your algorithm because yours makes tests which are known to be unneeded.
for candidate in [2,3,5,7,11,13,17,19,23,29,31,37,41]: while n % candidate == 0: n //= candidate primes.append(candidate)
If you have values larger than this then your assumptions (“the density of primes is high”) will no longer be valid and you would have to go to some other solution. Again, that list is only about 6,543 values to cover everything up to 2**32, and it will be quite fast.
Page updated
Google Sites
Report abuse