Golf related things

The mathematics of golf course rankings (Incomplete, for now!)

Australian Golf Passport (AGP) has recently released their inaugural ranking of the Top 50 golf courses in Australia: https://australiangolfpassport.com/australian-top-50-list-2025/.  Welcomingly, and unusually, they provide a fair amount of detail about the mathematics behind the rankings in the associated three-part podcast that announced the release of the rankings. Naturally, because this is the internet,  you probably think that I'm going to tell you everything they got wrong with their methodology. And you would be correct, somewhat. But, in defense of Matt Mollica and Scott Warren it would have been, literally, impossible for them to find a perfect methodology to use. 

Don't believe me, believe a Nobel Laureate (but really, believe the mathematics)

Ken Arrow, the recipient of the 1972 Nobel prize in Economics, is known for many contributions to economics including the eponymous Arrow's impossibility theorem. To oversimplify, Arrow's theorem says that it is impossible to create a ranking system that will always produce  a reasonable ranking list. Or, to put it another way: Arrow tells us that all golf course rankings are bad. (As an aside, it doesn't stop there. Arrow also tells us that all electoral systems are bad, and that all methods of calculating the welfare of a society are bad, too.) 

So, Matt and Scott shouldn't be eviscerated for using a flawed methodology -- all possible methodologies are flawed. Although, of course, some are better than others -- and the method used by AGP is actually a pretty reasonable choice, even if it does have some flaws. 

Suppose that you need to commit to a ranking methodology before you've collected the data (as you should), and that you want the methodology to give a "reasonable" ranking for all possible data sets that you might receive. Arrow's theorem tells us that it is impossible to create an ordering that satisfies all of:

• Completeness:  Every course must be ranked relative to every other course. Ties are allowed, but saying "I'm not sure which of these two is better" is not allowed.

• Transitivity: If Royal Melbourne West (RM West) is ranked above Kingston Heath, and Kingston Heath is ranked above 7 Mile Beach, then it must be that RM West is ranked above 7 Mile.

• Non-Dictatorship:  The ranking system can't just be a dictatorial system where we just agree with, for example, Matt's ranking and ignore everyone else's input. 

• Pareto efficiency: If every voter ranks New South Wales above The Australian, then the overall ranking needs to rank New South Wales above The Australian. 

• Independence of Irrelevant Alternatives (IIA):   If Peninsula Kingswood South (PK South) is ranked below The National Gunnamatta in the world where Peninsula Kingswood North (PK North) exists -- which happens to be the world in which we live-- then PK South must also be ranked below Gunnamatta in the parallel universe where PK North does not exist. (If we want to get technical, there are a few different versions of IIA, and Arrow's theorem uses a slightly different form of IIA than the one I used here. But, the differences are not important here, and this version is more intuitive for our purposes).   

The proof of Arrow's theorem is, as far as these things go, not *too* complicated. You can see two versions on the Wikipedia page for the theorem, if you are interested: https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem.

Independence of Irrelevant Alternatives

IIA is usually taken to be an obviously good property for a ranking system to have. Consider voting, one of the most common forms of ranking systems. In the 2000 US Presidential Election Republican George W Bush earned a very narrow victory over Democrat Al Gore, who was well known as being an early proponent for action on climate change. US elections are typically portrayed as being purely two-horse contests, but the Greens candidate Ralph Nader received almost 3% of the vote in the 2000 election. While we can't say for sure, it seems likely that a large proportion of Nader voters would have, instead, voted for Gore if Nader had not run for election. Thus, it is highly plausible that the mere existence of Nader changed the outcome of the election from Gore winning to Bush winning. This is a classic example of a failure of IIA, and it is rather uncontroversial to claim that an "ideal" election system should not exhibit this sort of failure of IIA. 

On the other hand, its not exactly obvious that a ranking of golf courses should satisfy IIA. In fact, it often seems reasonable that IIA should be violated when it comes to golf course rankings. One thing that makes a golf course standout, and hence become better rated, is if it is of a substantially different character to other golf courses. Lonsdale links is so very different to every other course in Australia, and this makes it special. If a clone of Lonsdale Links was constructed at Sorrento, then Lonsdale wouldn't be as special any more. So, would it be reasonable for Lonsdale's ranking to depend on the existence of this hypothetical Sorrento? For a more esoteric example, should the ranking of the new Lido course be affected by the fact that *the exact same course* used to exist 100 years ago on land that is more than 1000 km away from the current Lido?

So, its not all that clear that a golf course ranking *needs* to satisfy IIA, even though some people would prefer if it does. 

So, what did AGP actually do? The Plackett-Luce model.

The Plackett-Luce model is a common statistical model for aggregating rankings information. The intuition behind the model is pretty straightforward, although the details are highly technical. In the context of golf course rankings, the model assigns each course a score, and the ranking can be constructed by ordering the courses from largest to smallest score.  When you consider a pair of courses, the scores can be converted into a probability: the probability that course A would be ranked higher than course B. The Placket-Luce models assumes that IIA always holds, and it makes very strong use of IIA to in generating the scores. 

The model assigns scores by looking at the data and finding the scores that provide the best approximation -- called the "best fit" -- of the data. If exactly half of the raters thought course A was better than course B, and the other half course disagreed, then the best fit to that aspect of the data would be to assign the same score to each course. Next, suppose that a vast majority of people had course B ranked above course C: clearly B needs a much higher score than C. But, it might be that A and C were roughly evenly split -- about half of raters prefer C over A, so that A and C need roughly equal scores. The model certainly cannot accommodate all three requirements at once, so it selects a "best fit" that finds a compromise between these conflicting data points. In this case, it would likely do so by giving B the highest score and C the lowest score.

[How could the example in the last paragraph happen? Suppose that half of the raters rank the course {A,B,C,D} and the other half rank {D,B,C,A}.]

One really nice feature of the Plackett-Luce model is that it doesn't require very voter to provide a complete ranking list. If one persons ranks {A,B,C,D} but another has only played course B and D, then the second person can submit a ranking of {D,B} and the model will handle it just fine. In this case, the model will implicitly place more weight on the first persons list than the second person, because the first person effectively provided six pairwise rankings compared to one from the second person. 

The "original" Federal Golf Club layout

Recently, someone at Federal found some copies of an old course history, covering the period 1933 through 2011, titled The Federal Golf Club Story. There is no author listed, although a committee including Graham World, Terry Wingrove, Heather Oram, Paul Arnold, Barney McCabe and Stan Flaherty are thanked as contributors to the work. 

Remarkably, the book contains a 1927 routing of "Red Hill Golf Links" that was produced Alex Russell. Russell is famous for his work constructing Royal Melbourne's West Course and designing the East Course. Russell (clearly a busy man) had worked as private secretary for Prime Minister Stanley Bruce, and it was Bruce who had invited Russell to layout a course in Canberra for Bruce to play on. 

Russell's 1927 routing was never built. Nevertheless, 21 years later, a golf course was built on the same site. The 1948 course, originally just 9 holes but expanded to 18 in 1951, with some changes over the years, is the current Federal Golf Club. The current routing appears to have been developed completely independently of Russell's original layout.

The Federal Golf Club Story contains a line drawing of Russell's routing, and mentions that green sketches are available for nine of the holes at the National Archives of Australia (National Archives of Australia: Series A 6269, Barcode 241294). These files have been digitized, which appears to be rather uncommon for files of that vintage, and are readily available for viewing online.

Russell's routing, pictured below, clearly indicates the scale and orientation of his layout (note that north is to the right of screen). Armed with this, and his green sketches, I set about trying to determine precisely where his routing sat and how it relates to the current course. The nine holes depicted by solid lines with marked distances were intended to be built first, with the remaining holes to be constructed later. The original 9 holes would appear to be the 1st, 2nd, 5th through 9th, 11th and 12th holes of the full 18 hole course, but in what follows I will refer to them as 1 through 9, starting with the hole that heads southwest away from the clubhouse.

The two markers that I used to try and position the course are the fairway bunkers on the 1st hole and the 8th green. The bunkers on the first hole are described as being "cut into the face of a ridge", and I determined this ridge to be out to the right of the current 9th fairway. Russell's 8th green is depicted to be quite flat, and yet sits on what is generally quite sloped terrain between the current 11th and 12th holes. However, there is a section of land between those two holes that is both flat and perfectly shaped and sized for Russell's circular 8th green. Given that both of these points appear to be in agreement with each other, I am reasonable confident that I have located the course correctly.

[Aside: I can't work out what the "netting fence" running north-west from the clubhouse and between the 9th and 10th fairways is intended for. It's possible that this was an already existing fence line in 1927.] 

Amazingly, and in what appears to be a coincidence, the clubhouse for Russell's design sits almost exactly where the current Federal clubhouse sits today (the original Federal clubhouse was located near today's 11th green). An image of the modern course is shown below, followed by the two routings overlayed on top of each other. To orient the reader who is not overly familiar with the modern layout, the 1st and 10th holes both play out to the west of the clubhouse (towards the top of screen). The 1st hole, and front 9, sit to the south of the 10th hole, and back 9. The land generally slopes down from the east to the west, and less sharply from the north to the south.

It is immediately apparent that the Russell routing would be a much easier walk than the current layout. While Federal is not the hilliest course I have played (shout out to Furry Creek, BC, Canada, or, in Australia, the now closed Olinda course outside Melbourne), it certainly has some steep climbs up the 4th, 9th, 15th and 18th fairways.

The Russell layout achieves this by not going down to the low point at the at the current 3rd green, not climbing to the heights of the 4th green and 5th tee, not climbing up to the 14th green and 15th tee (the highest point on the current course), and having the finishing hole approach the clubhouse from the north rather than the much steeper west side.

The downside of the Russell layout is that it looks like it would miss out on some of the best views across the Brindabella Range that are afforded by the current course. The best view on the course is from the 4th green, which is entirely bypassed by Russell. The current 17th also has a great view after you crest the ridge in the fairway which might also be captured by Russell's 10th hole, although the subtly different angle of Russell's 10th might lessen the vista. It is possible that Russell's 2nd hole could also give a great view, although it is hard to tell given the modern tree lines.

Russell's layout would retain the essential character of the modern course: most approach shots are played with the ball either above or below your feet. Below, I present Russell's green sketches along with a few comments on how  I think the holes might play.

Hole 1 -- 387m

The 1st hole plays from the back room of the current pro-shop across the 9th fairway to a green near the current 5th. A shot that clears the fairway bunkers, positioned into a slight ridge about 120m from the green, would likely get a large kick forwards and run a long way. At 270m from the tee, these bunkers are well positioned to challenge the modern (very) long hitting golfer. I find it hard to see the strategic value of these bunkers in 1927, as they seem too far out to affect the tee shot and too short to affect the second. Nevertheless, I imagine they would frame the hole nicely. Notably, these are the only two fairway bunkers on the course that are specified by Russell.

Hole 2 -- 157m

The 2nd hole is a par 3 that plays downhill off the tee to a relatively flat green site. Russell clearly intended to build interest into the green and bunkering on this hole.

Hole 3 -- 331m

The third hole is a dogleg left that plays with a slightly downhill teeshot with a steeply uphill approach. The green site tilts back to front and right to left. A pulled approach shot that took a bad bounce could easily run right across the 4th fairway and onto the 5th, as the drop off to the left of this green is severe. This hole is probably a bit short by modern standards, with most players driving through the dogleg and into the up slope in front of the green. But, there does appear to be some room to move the tee back.

Hole 4 -- 351m

The 4th hole is a Par 4 with a slight dogleg right with a downhill tee shot and slightly uphill approach. The green is positioned on the left side of the approach of the current 7th fairway. Russell has indicated that the bunker in front of the green should be constructed on top of a man-made mound (or, less likely, it could have been a natural mound that has been flattened at some point during the last 100 years). The bunker position, well short of the green, would like make depth perception on the approach difficult.

Hole 5 -- 291m

The 5th hole, completing a sequence of three holes that run almost side-by-side, is a short par 4 that has a steeply uphill approach. A drive with a 240m carry would likely hit the up slope and stop or even roll back, with a 260m carry required to bounce up onto the green. A shorter drive would leave a tricky pitch, requiring either good spin control off an uphill lie or an accurate running approach between the bunkers.

Hole 6 -- 106m

The 6th hole, a short downhill par 3, looks to be one of the most interesting in Russell's layout. The entire hole, including the green, tilts heavily from right to left. My initial reaction to this hole was that it intended to play as a Redan hole, but that was before I looked at the length. There are some great pin locations at both the front of the green and behind the bunker. Even with a wedge in your hand, I imagine it would be hard to stop the ball to a left hand pin. Perhaps the play would be to try a running shot into that left hand pin, even though the hole is barely more than 100m long?

Hole 7 -- 235m

Despite being only 237 metres long, this hole is depicted as a dogleg in Russell's layout. This would be a fascinating hole to play with modern technology. The green is slightly above the tee box, with the hole running slightly downhill off the tee and then uphill into the green. The green is angled off to the right inviting a long fade off the tee. But, the green site also slopes heavily from left to right, pushing balls towards the row of 4 bunkers to the right of the green. I could imagine hitting anything from driver through to 4 or 5 iron off the tee here.

Hole 8 -- 390m 

The approach to the 8th hole plays uphill to flat green site. I could see this hole, with old hickory clubs, playing as either a long par 4 or short par 5. Although, given the flat, circular, green with a 30 yard radius I imagine that it was intended as a long 4. This is, for mine, probably the weakest hole of the nine. There is room behind the tee to lengthen the hole into a par 5 which, coupled with an improved green complex, could make the hole quite interesting.

Hole 9 -- 406m

The 9th hole is an absolute brute. It sits along the corridor of the current Federal 18th hole, which was originally a par 5 before the green was moved into its present location. Despite now being a par 4, most golfers cannot reach the current 18th in two even with modern clubs. With hickory clubs, Russell's 9th would have been a testing three shot hole. The hole runs steeply uphill, with the fairway falling off to the right as well and punishing any shot with even a hint of slice. The bunker cut 55m short of the green would be a formidable hazard for most golfers. 

Final thoughts

I figure the original 9 holes of Russell's layout to be a 2654m par 35 course. Perhaps a bit short by modern standards, but a remarkably easy walk and with several interesting holes. There would be space to extend a few of the holes to keep up with changes in technology over the last century, although the spacing between 3 green, 4 tee, and 5 green would probably be considered dangerous today. All in all, it is fascinating to see what might have been had the original layout been given the go ahead.