RPI: Non-Conference RPI
[NOTE: In 2022, a new NCAA rule eliminated overtimes except in conference tournament and NCAA tournament games. Unless stated otherwise, information on this page is based on data from 2010 to the present, with game results adjusted as though the 2022 change to no overtimes had been in effect. These game results adjustments are possible because the NCAA data system since 2010 has shown which games were decided in overtime. Also, the Covid-affected 2020 season is excluded from the data.]
Updated April 2024
The Non-Conference RPI (NCRPI) is a variation of the RPI. It is one of the resources available to the Division I Women's Soccer Committee in its NCAA Tournament at large selection and seeding process.
CALCULATION OF THE NCRPI
The NCAA computes the NCRPI using the same basic format as for the NCAA RPI. For information on the basic NCAA RPI format, go to the "RPI: Formula" page. The details of the NCRPI computation for Team A are:
Element 1: This is Team A's winning percentage in non-conference games. In other words, in computing Element 1, the NCAA disregards a team's conference games. It's worth noting that on average, teams play 18.7 games per year (including conference tournament games), of which 10.4 are conference and 8.3 are non-conference. This means 55.6% of games are conference and 44.4% are non-conference games, more for some teams and fewer for others.
Element 2: This is the average of Team A's non-conference opponents' winning percentages against teams other than Team A. In identifying the non-conference opponents' winning percentages, the NCAA includes all the non-conference opponents' games, both conference and non-conference.
Element 3: This is the average of Team A's non-conference opponents' opponents' winning percentages. In identifying the non-conference opponents' opponents' winning percentages, the NCAA includes all the non-conference opponents' opponents' games, both conference and non-conference.
Unadjusted NCRPI: As with the basic NCAA RPI, these three Elements combine to form the unadjusted NCRPI using the same formula as for the NCAA RPI:
NCRPI = (Element 1 + 2*Element 2 + Element 3)/4
Bonus and Penalty Adjustments: The NCAA uses an adjusted NCRPI. It uses the same bonus and penalty adjustment structure and amounts that it uses for the adjusted NCAA RPI. As it does in computing the bonuses and penalties that result in the adjusted NCAA RPI, the NCAA bases Team A's adjustment amounts on its opponents' unadjusted NCAA RPI rankings (and not on the opponents' unadjusted NCRPI rankings). For information on the bonus and penalty structure and amounts, go to the "RPI: Formula" page.
THE NCAA'S STATED REASON FOR USING THE NCRPI
A January 23, 2009 Memorandum from the NCAA's Associate Director of Statistics to the Division I Men's Basketball Committee provides the NCAA's publicly stated rationale for using the NCRPI for basketball. This is the only publicly stated rationale for using the NCRPI that I've been able to find. The Memorandum states:
"Teams and conferences are ranked by two methods. One includes all Division I games, the other is based on non-conference Division I competition only. Since conferences do not have the same number of members, a non-conference rating better evaluates the strength of each conference. For example, if a conference has 10 members and plays a [double] round robin schedule, then 18 of each of its teams' 28 games (180 total conference games) will produce virtually the same won-lost percentages in Factors II and III. But a conference with six members will produce the same numbers for only 10 of each of its teams' 28 games (60 total conference games)." (Emphasis added.)
For Division I women's soccer, the number of conferences' regular season games range from 7 to 11.
As I read the NCAA's explanation, it suggests that the main purpose of the NCRPI is to evaluate conference strength. Indeed, the explanation states that the NCRPI is better at evaluating conference strength than the RPI. The NCAA produces two conference ranking reports: League Ranking (all games) and League Ranking (non-conference). The League Ranking (non-conference) report gives the conferences' NCRPI data and rankings. Since the above NCAA explanation says that the NCRPI League Rankings are the better measure of conference strength, this means the NCAA is saying the Committee, in evaluating conference strength, should pay attention primarily to the League Ranking (non-conference) report.
It also is possible to read the NCAA's published rationale for the NCRPI as suggesting that the NCRPI is not really for evaluating individual teams.
IS THERE ANOTHER REASON FOR USING THE NCRPI?
Whatever the NCAA's stated reason is for using the NCRPI, there's another rating problem it might help with:
The Problem. Assume there are two 12-team conferences, each of which plays a full round robin among its members. Assume also that the game outcomes are such that in each conference, the conference winner has gone 11-0, the second place team has gone 10-1, the third place team 9-2, the fourth place team 8-3, and so on all the way to the basement dweller who has gone 0-11. Assume also that teams play only their conference games. In this scenario, the ratings for teams from the two conferences will match exactly. In other words, the conference winners will have identical ratings, the second place teams will have identical ratings, and so on. This will be true no matter what the actual relative strengths of the two conferences. One conference could consist of the top 12 Division I teams and the other conference could consist of U8 recreational soccer teams, yet their ratings will be the same.
This shows a characteristic of results-based rating systems, which is that they cannot differentiate the strengths of different pools of teams unless there are direct (Pool A plays Pool B) or indirect (Pool A plays Pool B plays Pool C) crossover games between the pools. This is the "playing pool" problem. Further, the reliability of the rating systems' comparisons of the relative strengths of different pools depends on the number of crossover games: the more crossover games there are, the more reliable the comparisons, and the fewer the crossover games, the less reliable the comparisons.
For conferences and the NCAA RPI, what this means is that (1) the NCAA RPI starts out treating all conferences as of equal strength and (2) this is a problem that non-conference games mitigate. Whether due entirely to this problem or also to other problems with the NCAA RPI's structure, however, the NCAA RPI does not properly rate teams from different conferences in relation to each other. The RPI: Measuring the Correlation Between Teams' Ratings and Their Performance page demonstrates this. The NCAA NCRPI, by disregarding altogether a conference's intra-conference games, should help. The question is, does it and, if so, how much?
HOW DOES THE NCAA NCRPI HANDLE THE "CONFERENCE PLAYING POOL" PROBLEM?
In order to see how the NCAA Non-Conference RPI addresses the conference playing pool problem, I use my Correlator method of analysis described on the RPI: Measuring the Correlation ... page. If you have not reviewed that page before getting to this one, I recommend you review it before continuing here as I will use terms here that I define on that page.
Performance Percentage Analysis
The following table shows the average the NCAA NCRPIs of the conferences' teams in relation to their performance percentages. The conferences are arranged in order from the conference with the best average NCAA NCRPI to the conference with the poorest:
This table shows that for all games, the NCAA NCRPI High/Low Spread for All 10% Segments is 11.7% and the Over/Under Total is 77.0%.
The following chart, based on the table, shows the relationship between conferences' average NCAA NCRPIs and conferences' performance in relation to those averages:
Based on this chart, the Trend Spread for all games is -1,7%.
Comparing the NCAA RPI to the NCRPI
Comparing the results for these two systems:
Keeping in mind that the closer to 0% any of these numbers is the better, this comparison shows that, for fairness at rating conferences in relation to each other, the NCRPI is better than the RPI. It is significantly better in general fairness (the Spreads and the Over and Under Totals) and far better in terms of discrimination based on conference strength (the Trend Spreads). In my opinion, the trend spreads are close enough to zero to mean that the NCRPI does not discriminate significantly based on conference strength.
Actual Results Compared to Likely Results Analysis
The following table shows the average NCAA NCRPIs of the conferences' teams in relation to their Actual Results Compared to Likely Results Winning Percentage differences. The conferences are arranged in order from the conference with the best average NCAA NCRPI to the conference with the poorest:
This table shows that for all games, the adjusted NCAA NCRPI High/Low Spread for Actual Less Likely Winning Percentage is 8.5% and the Over/Under Total is 44.4%.
The following chart, based on the table, shows the relationship between conferences' average NCAA NCRPIs and conferences' Actual Results Less Likely Results Winning Percentage differences in relation to those averages:
Based on this chart, the Trend Spread is 4.9%
Comparing the NCAA RPI to the NCAA NCRPI
Comparing the results for these two systems:
This table shows that the NCAA ANCRPI does better than the NCAA RPI, in rating teams from conferences in relation to teams from other conferences. It shows a lesser improvement, however, than the Performance Percentage Analysis method shows.
HOW DOES THE NCRPI HANDLE THE "GEOGRAPHIC REGION PLAYING POOL" PROBLEM?
As shown on the RPI: Measuring the Correlation ... page, in addition to having a problem properly rating teams from a conference in relation to teams from other conferences, the NCAA RPI has a problem properly ratings teams from a geographic region in relation to teams from other geographic regions. How does the NCAA Non-Conference RPI do with regions?
Performance Percentage Analysis
The following table shows the average NCAA NCRPIs of the regions' teams in relation to their performance percentages. The regions are arranged in order from the region with the best average NCAA NCRPI to the region with the poorest:
This table shows that for All 10% Segments, the NCRPI High/Low Spread is 19.4% and the Over/Under Total is 28.1%.
The following chart, based on the table, shows the relationship between regions' average NCRPIs and regions' performance in relation to those averages:
Based on this chart, the Trend Spread for all games is 14.9%.
Comparing the NCAA RPI to the NCAA NCRPI
Comparing the results for these two systems:
Keeping in mind that the closer to 0% any of these numbers is the better, this comparison shows that, for fairness at rating regions in relation to each other, the NCAA NCRPI is better than the NCAA RPI in some respects and poorer in others.
Actual Results Compared to Likely Results Analysis
The following table shows the average NCAA NCRPIs of the regions' teams in relation to their Actual Results Compared to Likely Results differences. The regions are arranged in order from the region with the best average NCAA NCRPI to the region with the poorest:
This table shows that for Actual Less Likely Winning Percentage difference, the NCAA NCRPI High/Low Spread is 8.0% and the Over/Under Total is 12.0%.
The following chart, based on the table, shows the relationship between regions' average NCRPIs and regions' Actual Less Likely Winning Percentage differences in relation to those averages:
Based on this chart, the Trend Spread for all games is 5.9%.
Comparing the NCAA RPI to the NCAA NCRPI.
Comparing the results for these two systems:
Keeping in mind that the closer to 0% any of these numbers is the better, this comparison shows that the NCAA RPI does a little better than the NCAA NCRPI, at handling the geographic region playing pool problem. In general, however, the differences are small.
IS THE NCRPI USEFUL FOR RATING INDIVIDUAL TEAMS?
A trade off for the NCRPI handling the conference playing pool problem better than the NCAA RPI is that the NCRPI ratings' correlations with game results are significnatly poorer than for the NCAA RPI. The following table shows this:
This table shows how well the systems' ratings, as adjusted for game locations, correlate with actual game results. The "Overall" columns show that the NCAA RPI is significantly more accurate than the NCAA NCRPI, when looking at all games. The "Top 60" columns show that the NCAA RPI likewise is significantly more accurate than the NCAA NCRPI, when looking only at games involving at least one Top 60 team. In comparing rating systems, these differences in rates of correlation between ratings and game results are large and suggest the NCAA NCRPI is unacceptable for rating individual teams. This fits with the NCAA's rational for using the NCAA NCRPI, which is that it rates conference strength better than the NCAA RPI does -- whereas the NCAA makes no claim that it is useful for rating individual teams.
DOES IT ALL MATTER? HOW MUCH DIFFERENCE DOES THE NCRPI MAKE IN THE RANKING OF CONFERENCES?
Given that the NCAA's rationale for using the NCAA NCRPI is that it "better evaluates the strength of each conference," how much difference does it really make in the ranking of conferences? To answer this question, here is a table that shows the rank of each conference by average NCAA NCRPI and by average NCAA RPI since 2010:
As you can see, among the top 12 conferences there are, at most, changes of 1 rank position. Further, if you look at the 1 rank position changes, the NCAA RPI rating differences between the conferences are small. For practical purposes, these are the conferences in contention for NCAA Tournament at large selections and seeds. Thus for practical purposes, although the NCAA NCRPI may be better at ranking the conferences (but not the regions) in relation to each other, for Division I women's soccer it is not enough better to add significantly to the NCAA RPI as a tool for evaluating conference strength.