Umpire Development in Australia

Questions about the history and context of SAL umpiring assessment methodology particularly with regard to the relationship between an umpire’s pre-game mood and in-game performance along with any psychometrically assessment for validity or reliability has prompted the opportunity to further document SAL’s umpiring framework.

Australian Beginning

Organised Softball competition commenced in Melbourne in 1942.  Increasingly competitions were played in Sydney, Adelaide and Brisbane and by the end of 1946 four State Associations were formed.  The first interstate competition was played for the first time in 1947 and officiated by baseball umpires with an interest in Softball.  

The Melbourne competition was growing and developing a strong base of players, coaches and umpires.  The 1951 National Championship also demonstrated that Adelaide was producing competitive players and capable umpires. 

In 1953 an All Australian rule examination was introduced and umpire performance assessed at the 1954 National Championship leading to two Victorian, two New South Wales and one South Australian being accredited.  Records are not available to show who conducted the first performance assessment???  

Over the next six years, six more Victorians, six South Australians, two from Western Australia and one from Queensland were added to the accredited umpire list.  New South Wales did not gain another accredited umpire until 1960.  An additional twenty umpires were accredited during the 1960s including eleven Victorians.  In 1968 accreditation went to an umpire from the Australian Capital Territory and Tasmania for the first time.  

Australia provided eight umpires for the first International Softball Federation Women’s Championship played in Melbourne in 1965.  Eight Australian umpires officiated the Championship, six from Victoria and two from South Australia.  The event provided signs that the Australian umpires’ interpretation of the pitching rule was different from the Americans when the iconic USA pitcher, Bertha Tickey, was called illegal.  

The evidence shows the dominance of Victoria within the national umpiring scene. The group placed a strong emphasis on rules knowledge and interpreting how the rules should be administered including on-field mechanics. The isolated thinking fuelled State and personal rivalries between the main personalities of the day. 

Over time, Australian umpire assessors had access to baseball assessment documents as well as those used by the American Softball Association (ASA).  ASA assessment was influenced by their senior umpires who had also been part of major and minor league baseball.  The ASA documents listed a range of items to be considered from dress code, positioning to calling balls and strikes.  

The Australian umpire assessors, predominately Victorian, developed a framework that they considered met Australian needs.  An umpire’s on-field performance was observed and notes recorded things that an umpire did which may lead to an incorrect decision.  This could be to do with positioning, calling too quickly, appearance, confidence in making a call, knowledge of rules, calling of illegal pitches and........ sometimes it appeared, on judgement.  Marks were deducted for the notes recorded and some allowance made for repeated issues although some types of repeats would cause a candidate to fail for that particular reason.

On field Assessment at National Championship level.

Points to be noted:

Candidates understood that there were a set of games that they would be examined on.

• Six games were used in the assessment – three plates and three bases.

• There was a pre-championship umpire meeting.  It was primarily to receive the Championship handbook and the game allocations.  Off field behaviour and alcohol consumption was mentioned. There was no pre-championship direction re mechanics etc.

• In the main umpires were NOT required to stay together – addresses and contact numbers of accommodation [hotels, friends or family] were provided to tournament officials.

• Notes taken for the on-field performance were grouped into five categories:

APP = Appearance

DEM = Demeanour

COG = Control of Ground

GIP = Getting Into Position

MOC = Method of Calling.

• Each was worth 20pts and one steadily lost points for real or perceived errors. 

o   The umpire lost points for an error that could have led to a possible incorrect application of a rule or a position that might have consequences if another play had occurred.  Many considered this grossly unfair. 

• It was a system that promoted uniformity and did not cope well with simple personal differences such as a 6 feet tall umpire needing to be a slightly different distance from a play to a 5 feet tall umpire.

• There was no feedback given after games, thus no opportunity to correct perceived mistakes.

• The first an umpire knew of any shortcomings in any of the listed areas was upon receipt of the pages of notes when advised of the result at the assessment interview.

In parallel States developed their own accreditation system, perhaps along the lines of the national model.  Given the subjective nature of the performance recording and the disproportionate distribution of accredited umpires who understood the system, State assessments were essentially their own, unique, accreditation system that bore little or no resemblance to one another. 

An extra dimension for the emerging umpire was that they often needed to gain an umpiring accreditation from their local association before being eligible to be assessed at State level.  These local assessments were independent and further removed from any standardisation.   

The State and local standards / requirements had to be forgotten and, significantly, new standards / requirements learned once the umpire arrived at the National Championship, not ideal for maximum performance.  There was no ‘national’ thinking as far as umpiring the game was concerned and tensions were exacerbated when any State sought rule interpretation from the ASA Umpire-In-Chief and incorporated the advice in their State program. 

Umpires sent with Australian teams to world championships and other international tournaments also found themselves in an identical position of change, needing to re-learn umpiring mechanics and critical rule interpretations during the warm up games or opening games of the international event.  

On field, Australian umpires needed to learn about making calls from inside the infield and about rotation to cover their partners when they covered the deep outfield.  Both these elements were missing from the Australian system.

A related point about the disparate State and local umpire assessment systems was that as late as the early 1980s the National Umpire in Chief counted only Australian certified umpires as ‘umpires’ suggesting that Australia at most times had less than 30 active umpires.  Also, it was not uncommon for an umpire to cease umpiring once they gained their Australian accreditation.

By 1980 there was an increasing impetus to bring about an immensely different approach to officiating the game in Australia.  It was driven by different aspects of our sport.  

In general, by the players, in particular, national coaches because of what was happening on the field to our national teams and, to a lesser extent, what our umpires on overseas assignments observed and experienced – definitely not to their advantage in both cases.

The issues reached a head at the 1984 Australia Games when the Federation gained Men’s Softball in the event.  An exhibition game against New Zealand was one of the last events on the entire Australia Games schedule and attracted the presence of the Minister for Sport.  The exhibition game was marred by Australia’s interpretation of the illegal pitch and the disruption these calls had on the game. 

The Australian Softball Federation chose not to re-appoint the Umpire-In-Chief a month later and instead to build a new umpiring administrative structure.

The new administration needed to address the short falls of the previous system which included

• Under estimating the number of umpires in Australia

• Developing an umpiring structure that catered for umpires from local competition to national level

• Creating an assessment methodology that had greater documented substance and that was transparently fairer

• Implementing standards that allowed consistency of application across all levels of competition

• Promoting the Australian umpiring system being consistent with international rule application and on-field mechanics

It is reasonable to say that the pre-1985 Softball Australia umpiring assessment methodology did not consider the relationship between an umpire’s pre-game mood and in-game performance.

Post 1985

The 1985 change was marked by a re-titling of the head of the umpire program from Umpire-In-Chief to Director of Umpiring and allocation of the task to address the inherent problems described. 

Those commissioned with the change agenda set about defining the scope, the priorities and where assistance could be sought.   Australian teams had typically prepared for American hosted competition on the West Coast and often in Vancouver.  Many of our world championship umpires were assisted by British Columbia umpires to transition from their Australian mechanics to those needed to be effective during international competition.  The Canadians produced quality resources on umpiring mechanics as did the ASA.

Beyond historical support from the Canadians, there were some fundamental differences between the Canadian and United States (ASA) umpiring programs.  ASA had more umpires than Australia had players.  ASA ran great umpire schools but USA did not have a national levelled umpire system.  By contrast the Canadians had developed a five level national accreditation system.  Australia commenced building its revised umpire program on the Canadian model as it incorporated the Australian needs of accrediting umpires as local, State and National level. 

In the winter of 1985, the Acting Umpire-in-Chief convened a meeting of senior State umpiring administrators to plan the way forward.  The New Zealand Umpire-In-Chief was also invited to provide input about their umpire program.  A few months later an umpiring school that included targeted umpires from all States was held in Sydney. The change period was an extremely difficult time for all, with personality clashes a recurring feature of the change.

An assessment system that followed the Canadian model was developed.  The assessment was based on six games and included, for each game, over twenty items that were grouped into five broad categories – General, Game Control, Rules Knowledge / Judgement, Positioning and Plate Work. 

The items allowed ranking from 1 to 5 with 4 being the standard.  If an item was not highlighted in a particular assessment it was scored 4.  If incident caused the standard not met the score was reduced to 3 and if there were multiple similar incidents the score reduced to 2.  In exceptional circumstances when the umpire did something outstanding to do with a particular item, 5 was awarded. Each of the 20+ items were so treated for each game assessed, leaving out the Plate Work assessment on base allocation games.  The raw 1 to 5 scores were summed in each category score. 

The category scores were then proportionally combined and summed for a game score.   The total games scores then determined if the accreditation level was met.  While the methodology appeared a little complicated initially, it has proven extremely successful over a long period of time and the use of laptops hides the mathematical genius. 

Proportioning the raw category scores is the core to the success and allowed ready change as the umpire program emphasis matured and changed.  The initial implementation placed highest value on the traditional judgement and rules knowledge but within a few years plate mechanics, positioning and rules / judgement were all treated with equal importance in weighting the assessment outcome.  Also, items could easily be moved between categories on the few times this was deemed necessary. 

Other benefits of the method were that it included a consistent assessment basis for umpires at all levels. 

It provided for entry level umpires, Level 1s who were critical to grass roots competition.  Levels 2 to 4 were available for State and local associations to award.  Level 5 and 6 for National junior and open competition. 

The same assessment system reviewing the same 20+ items of umpire performance were used.  The need to make the pre-1985 mechanics transitions had passed. 

In 1994 a two days examiner training course was developed and delivered to all examiners attending 1995 Championships.  The participants attended from all States and Territories except Tasmania and the Northern Territory where there were no suitable personnel available.   The course made a major contribution to standardising the approach to assessment at national level and directly improved State based assessment practice.

Being able to count all levelled umpires in the system was extremely important for the Sport and more accurately counted the hundreds of umpires that were serving the game each week.

By the mid-1990s the assessment elements were being applied to all umpires who attended a national championship and provided a firm basis for maintaining individual performance and ensuring that maturing umpiring mechanics necessitated by the evolution of the game were being adopted by certified umpires.

The annual umpires’ theory examination also changed after 1985.  The hundred questions narrative answers were changed to multiple choice.  This better evaluated the umpires rule knowledge, separating that from their writing skills and greatly reduced the marking procedure. 

Summary

The comments provided explain in some detail on the history of how the method SAL examination of umpire performance developed.  It was never psychometrically assessed for validity or reliability. 

The 23 items in the assessment have varied by one or two over time and have their origin in ASA documentation that dates back prior to 1980 and most likely come from Major League Baseball umpire schools.   The five overall categories appear to have been in the baseball / softball literature for a very long time with varying names. 

The five categories used in Australian Softball umpiring before and after 1985 could at best be considered roughly similar in their underlying elements although not identical.  Since 1985, the category terms migrated to those in use today and were agreed at State Directors of Umpires meeting with subsequent SAL endorsement.

On the core question of evaluating an umpire’s pre-game mood to in-game performance, there is no evidence that this assessment has ever taken place.

Compiled by:

Ken Culpitt

Alan McAuliffe

Margo Koskelainen

September 2017.