RPI: Home/Away/Neutral Issues

Updated February 2015

As discussed on the "RPI Formula" page, in general the NCAA computes the RPI for Division I women’s soccer without regard to game locations.  The one exception is in the adjustments the NCAA makes to the unadjusted RPI in order to produce the Adjusted RPI.  The adjustments award bonuses for good wins and ties and impose penalties for poor ties and losses, with the bonus and penalty amounts depending in part on whether a team receiving an adjustment played the game at home, at a neutral site, or away.  Apart from these adjustments, however, the NCAA computes the ratings without regard to game locations.

Some critics have argued that the basic RPI formula should make a distinction between home, away, and neutral site games, in recognition that there is a home field advantage.  The critics have asserted, in particular, that the top teams and teams from the strong conferences have leverage that enables them to play more non-conference games at home than away and that this unfairly skews the RPI in their favor.

For Division I men's basketball, the RPI formula makes a game site distinction:  In its Element 1 (Team's winning percentage) computation, it treats a home win as 0.6 of a win and an away win as 1.4 wins; and an away loss as 0.6 of a loss and a home loss as 1.4 losses.  In baseball, there is a similar distinction except that the weights are 0.7 and 1.3, since home field advantage is statistically less in baseball than in basketball.  The question is whether Division I women's soccer should convert to a comparable system.  This would mean incorporating game location data into the basic RPI formula.

HOME/AWAY IMBALANCES

In order to test whether teams play balanced home/away schedules, I looked at conferences and regions over the 2007 to 2014 seasons to determine whether they have game location imbalances.  I found that they do.  I also found a pattern to the imbalances:  Conferences and regions with higher average Adjusted RPIs tend to have favorable home game imbalances and conferences and regions with lower average Adjusted RPIs tend to have unfavorable home game imbalances.  This is not true in all cases, but it is true on average.

The following table shows the relationship, for the 2007 through 2014 seasons, between conferences’ average NCAA ARPIs and conferences’ home game imbalances.  It is in order from the conference with the highest average ARPI to the conference with the lowest.


HAN Distribution for Conferences Table


The following chart, derived from the above table, shows the relationship between conference strength and game locations:

HAN Distribution for Conferences Chart

As the trend lines show -- pink for percent of non-neutral games at home and yellow for percent of non-neutral non-conference games at home -- there is a correlation between conference average strength and percent of games at home: stronger conferences' teams tend to play a higher percent of games at home than weaker conferences' teams.

The following table shows the relationship, for the 2007 through 2014 seasons, between the regional playing pools’ average NCAA ARPIs and the pools’ home game imbalances.  It is in order from the region with the highest average ARPI to the region with the lowest.

HAN Distribution by Region Table


The following chart, derived from the above table, shows the relationship between region strength and game locations:

HAN Distribution by Regions Chart



As the trend lines show -- pink for percent of non-neutral games at home and yellow for percent of non-neutral non-region games at home -- there is a correlation between regional playing pool average strength and percent of games at home: stronger regions' teams tend to play a higher percent of games at home than weaker regions' teams.

In summary, there are home/away imbalances.  Further, stronger conferences and regions tend to have favorable home game imbalances and weaker conferences and regions tend to have unfavorable home game imbalances.

HOME FIELD ADVANTAGE

IS THERE A HOME FIELD ADVANTAGE IN DIVISION I WOMEN'S SOCCER?

Given that there are home/away imbalances, the next question is whether there is a home field advantage and, if so, its extent.

In order to determine whether there is a home field advantage in relation to RPI ratings, I use my Correlator and performance percentage method of analysis applied to data for the eight seasons from 2007 through 2014.  (See the "RPI: Measuring the Correlation Between Teams' Performance and Their Ratings" page for information on the Correlator and performance percentage method of analysis.)  In a performance percentage analysis, a percentage above 100% means that the group of teams, on average, is outperforming its ratings; and a percentage below 100% means that the group is under-performing its ratings.

Specifically, I use the performance percentage method to compare teams' performance percentage in home games as compared to their performance percentage in away games.  I do this for all games regardless of the rating difference between opponents, and also for the most closely rated 5%, 10%, and 15% of games.  In the table below, I show the results of this analysis for the Unadjusted RPI.  I've chosen the URPI because it is "uncluttered" by the bonus and penalty adjustments and thus gives the cleanest picture of the extent of home field advantage:

HAN Extent URPI, New

Looking first at the two right-hand columns in the table, they say that for the most closely rated 5% of games, home teams perform better than their ratings (unadjusted for game location) say they should -- their performance percentage is 116.0% rather than the norm of 100.0% -- and away teams perform more poorly than their ratings (unadjusted for game location) say they should -- at the 84.0% level rather than at 100%.  The same is true looking at the most closely rated 10% and 15% of games, as well as for all games.  Based on other work I've done, the impact of home field advantage probably is best measured by the above 5%, 10%, and 15% numbers.  Those are the games in which teams are closely rated enough that the impact of home field advantage has a likelihood of affecting games' outcomes.  (For games with larger rating differences, the higher rated is likely to win regardless of game location)

Although the exact numbers vary, the above pattern is the same for every other RPI variation as well as for other RPI-based rating systems I've developed and for Massey's and Jones' rating systems.  Simply put, there is a home field advantage in Division I women's soccer; and the advantage affects results in a significant number of games.  This, of course, is not surprising.

WHAT IS THE EXTENT OF HOME FIELD ADVANTAGE?

Knowing that there are home field imbalances and that home field advantage affects game results in a significant number of games, the next step is to measure the effect.  Since home teams perform as though their ratings are higher than their NCAA RPI ratings and away teams perform as though their ratings are lower, this suggests that there should be an upward RPI adjustment one could add to teams’ ratings when they host games and a matching downward adjustment to teams’ ratings when they are visitors, such that, with those game-by-game adjustments, the teams then would perform as a whole in accord with their "Home/Away/Neutral (or HAN) Adjusted" ratings -- in other words, their performance percentages would be right around 100% for both home and away games.  With that in mind, for the eight years 2007 through 2014, for each rating system, I test a series of HAN Adjustment amounts, made on a game by game basis, to reflect game location.  This includes testing the Adjustments in games in which opponents are closely rated, to see what level of HAN Adjustment will produce correlations in which teams perform in accord with their HAN-adjusted ratings regardless of whether they are home or away.  I do this for each rating system I want to evaluate.

The HAN Adjustments I initially tested for the RPI variations ranged from +0.001 for home teams matched by -0.001 for away teams, to +0.015 for home teams matched by -0.015 for away teams.  In conducting the tests, I found that the results would converge on a particular set of matching upward and downward Adjustments at which teams’ performances matched their RPIs in closely rated games, regardless of game location.  For Adjustments of lesser amounts, home teams still outperformed and away teams under-performed their ratings; and for Adjustments of greater amounts, home teams under-performed and away teams outperformed their ratings.   I also focused on the most closely rated 5%, 10%, and 15% of games, since those are the games for which the teams are closely enough rated that game locations would have a reasonable chance of affecting game results.

The following table illustrates how this process works, using the Unadjusted Non-Conference RPI as an example:

HAN Adjustment Amount URPI, New


In this table, what I look for is the adjustment amount at which home and away team performance percentages are closest to 100% for the 5%, 10%, and 15% most closely rated games.  The adjustment amount I selected, highlighted in yellow, is 0.0090.  As the table shows, it takes some judgment in deciding on the right adjustment amount.  For the UNCRPI, possible candidates are 0.0080, 0.0085, or 0.0090.  My conclusion is that 0.0090 is the correct adjustment.  A lesser Adjustment leaves the home teams outperforming and the away teams under-performing their ratings; and a greater Adjustment leaves the home teams under-performing and the away teams outperforming their ratings.  Thus using the UNCRPI rating system, a home team on average performs as though its rating is 0.0090 higher and an away team performs as though its rating is 0.0090 lower than the rating actually assigned by the UNCRPI.  In other words, the change in UNCRPI rating difference when moving from a neutral site to a home field site is 0.0090 + 0.0090 = 0.0180.

The appropriate HAN Adjustment depends on the particular rating system and, for the RPI, on the particular RPI variation.  In the above table, in the left hand column's last three rows, I've indicated the outside rating difference for each of the most closely rated 5%, 10%, and 15% of games.  Thus the "outside" UNCRPI rating difference for the most closely rated 5% of games is 0.0054; for the most closely rated 10% is 0.0108, and for the most closely rated 15% is 0.0162.  You may notice that these are in multiples, in other words 0.0054 x 2 = 0.0108; and 0.0054 x 3 = 0.0162.  Although the numbers don't match up exactly this way for each rating system, the pattern nevertheless is consistently an approximate 1:2:3.

The different rating systems have different "spreads" in their ratings, so that some rating systems' numbers for the "outside" ratings discussed in the preceding paragraph are greater than those for the UNCRPI and some are less.  What this means is that ratings are "spread out" more in some rating systems than in others.  The needed HAN adjustments for home field advantage appear to be based on those spreads.  The following table, and chart based on the table, for the different RPI versions, demonstrates this:

HAN Adjustments and Systems Table, New

HAN Adjustments and Systems Chart, New


In the above chart, the uneven blue, pink, and yellow lines with the data points are the "outside" differences between opponents ratings for the most closely rated 5%, 10%, and 15% of games respectively.  The uneven green line with the data points are the HAN adjustments for the different versions of the RPI.  The straight green line is a computer generated straight trend line based on the uneven green line.  It shows that the correct HAN adjustment declines as the "outside" differences between opponents decline.  Since the HAN adjustment is applied upward to the home team and downward to the away team, the actual rating difference effect of home field advantage is twice the HAN amount.  Looking at the above chart, this puts the rating difference effect somewhere between the "outside" difference for the most closely rated 10% and the most closely rated 15% of games.

Here are the same table and chart for the Non-Conference RPI:

HAN Adjustments and System Table, NC, New

HAN Adjustment and Systems Chart, NC, New


I'll note that for the NCRPI variations, the rating difference effect of home field appears to be about the level of the "outside" difference for the most closely rated 15% of games.

ARE HAN-ADJUSTED RATINGS A BETTER PERFORMANCE MEASURE? 

HAN-adjusted ratings should correlate better with game results than ratings that have no HAN adjustments.  To see whether this actually happens, here is how ratings and game results worked for the URPI and UNCRPI, with and without HAN adjustments, using all games played from 2007 through 2014:


HAN Correlations Improvement URPI and UNCRPI

As the above table shows, the HAN Adjusted URPI and UNCRPI, as expected, correlate slightly better with game results than the URPI and UNCRPI without the HAN adjustments.  This same pattern of the HAN Adjusted ratings correlating slightly better with game results than the HAN unadjusted ratings occurs for all rating systems.  This is one of the reasons it's important to make game-by-game rating adjustments for game locations when I run the correlator.  It also is important because it assures that when looking at how conferences and regions perform in relation to their ratings, the results are not being influenced by game location imbalances.

Comments