Open post for questions about rankings

Let’s try something a little different this week — does anyone have any questions about rankings (PWR, probably being most interesting)?

The next two weeks are interesting because the schedules aren’t firm. Some conferences will begin conference tournaments, some have a week or two of regular season play left.

I’ll still try to make some regular posts later in the week, but this is your chance to find out what YOU really want to know.

Last weekend of February tournament cutlines

As we enter the final full weekend of regular season play (there is some regular season play next weekend, and the Big Ten pushes into the weekend beyond that, but over half the remaining regular season games occur this weekend), I want to remind readers that these forecasts will be through the end of the regular season only.

Conference tournaments don’t provide a lot of downside risk, because they tend to be single elimination (the notable exception being that it’s possible to go 0-2 in conference play in conferences with play-in series). However, there can be significant upside opportunity because teams in conferences with play-in series can put together something like a 4-1 run (a perfect record in conference play would earn the autobid, thus rendering the final PWR ranking unimportant).

Because of those games remaining to be played, I loosely define ending the regular season ranked 13-17 as “on the bubble”. Teams in those rankings can secure an autobid with a decent conference tournament performance.

#7 Denver is the highest ranked team with a decent chance of falling to the bubble if they slump.

denver

#10 Minnesota and below actually need to do pretty well (e.g. above .500) to avoid falling to the bubble (note this chart was made before last night’s win).

Minnesota

Former top-ranked #18 Harvard and below need good performances to climb onto the bubble.

Harvard

Though it’s unlikely that #23 Robert Morris will climb into contention, #24 Western Michigan, #25 Bemidji State, and #26 Penn State are long shots if they win out.

robertmorris

westernmichigan

bemidjistate

pennstate

#27 Dartmouth and below are unlikely to make the NCAA tournament without significant success in their conference tournaments.

dartmouth

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

A new #1 in KRACH

Unlike PWR (which mimics the tournament selection process) , KRACH is just for fun. But, a lot of people like it and it’s what I use to estimate team strength when simulating game outcomes.

When writing yesterday’s post, I noticed there’s a new king of the hill in PWR – #2 North Dakota.

Only once this season has #1 Minnesota State been knocked out of first place in KRACH, on Dec. 29 by then second-in-PWR Harvard. The following week Harvard also took over first place in PWR. Harvard’s reign was short-lived, as Minnesota State took back the top rankings in both PWR and KRACH on January 12 and have held both until this week.

krach

pwr

Big PWR games of the week

#10 Minnesota appears in the Big PWR Game of the Week for a second time. Buoyed by a road split last weekend, the Gophers probably need a better performance hosting #33 Michigan State to avoid falling back down to the bubble.

Getting swept could incite numerous “Time to get rid of the stupid PWR?” forum threads, as Minnesota would likely fall 8-9 places.

minnesota_oneweek

The runner-up, #22 Northeastern, could provide a shock in the other direction by appearing on the bubble if they sweep #6 Boston University.

northeastern_oneweek

Finally, #1 Minnesota State faces the biggest threat to its ranking in weeks with a series hosting #5 Michigan Tech. The Mavericks need a sweep to hold off #2 North Dakota.

mankato_oneweek

michtech

Big PWR game of the week

The big PWR game of the week is #14 Yale vs. #10 Quinnipiac. They only play one game vs. each other, but then another each vs. #56 Princeton and #48 Brown, respectively.

A single win this weekend for either most likely results in a small decline in ranking. The interesting outcome is if the loser of the head-to-head also loses their other game, which could result in falling of at-large bid position in the PWR.

yale

quinnipiac

The runner-up big PWR game is #23 Western Michigan vs. #2 North Dakota. North Dakota isn’t moving much, even if they get swept; it doesn’t seem possible to overtake idle #1 Minnesota State this weekend. However, Western Michigan could jump just below the bubble with a sweep.

westernmichigan

northdakota

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

Tournament cutlines and weekend PWR outlook

Welcome new visitors. You might want to start with my introductory post, Hello world, to see what this blog is about. It may not be for everyone.

Review of last week’s cutlines

I don’t report on the cutlines (the rankings above which teams are locks for the tournament and below which teams are unable to make the tournament at-large) weekly, because their movements are usually pretty intuitive. If I reported that a team needs to win 5 out 8 and it subsequently wins 2 games, it then needs to win 3 out of 6; the PWR curves usually look about same, just the curve labels change from “5 more wins” to “3 more wins” and so forth. To illustrate that, let’s quickly review a few of the teams that had charts in last week’s article (you may want to open its charts side-by-side for comparison if you can).

By winning 2 games, #4 Minnesota-Duluth made the old “win 0″ curve drop off and now just needs 1 or 2 more wins to stay on or above the bubble.

minnesotaduluth_endofseason

#5 Bowling Green also won 2 games, so now just needs about 4 wins to go into conference tournaments on the bubble.

bowlinggreen_endofseason

Further down the chart, #14 Minnesota shifted all of its curves with a pair of wins — the Gophers now need about 6 or 7 wins out of 10 (consistent with last week’s 8 or 9 out of 12) to climb onto the bubble before conference tournaments.

minnesota_endofseason

#30 Bemidji State, which I said last week could only afford about 2 losses, has racked up 2 losses. They would pretty much need to win out for a shot at an at-large bid.

bemidjistate_endofseason

Interesting potential movements this weekend

First, is this the week #1 Minnesota State falls out of first? It only seems possible if they get swept (which KRACH gives about a 2.6% chance of happening), and even then someone nipping at their heels (North Dakota seems the only possibility) has to do well. You can’t see the “Win 1″ curve because it’s in exactly the same place as “Win 2″—100% at 1.

mankato

The matchup of the weekend is definitely #12 Michigan vs #14 Minnesota. Neither has much upside potential, but if either sweeps the other will plummet up to 10 spots.

michigan minnesota

#15 Mass.-Lowell needs a sweep to hang on, but pair of losses could send them into the twenties.

masslowell

Remember when #16 Harvard was ranked 1st and I said that a “not particularly likely” bad 2nd half could still push them out? Two more losses this weekend could push them into the twenties.

harvard

#22 St Cloud State, mentioned last week as the lowest ranked team with a good chance of climbing into contention, can make up some ground this weekend. An unlikely sweep of #5 Minnesota-Duluth could catapult them up onto the bubble, while even a split could result in a climb of a position or two.

stcloudst

#26 Western Michigan is also poised for huge jump with an also unlikely sweep over #4 Nebraska-Omaha.

westernmichigan

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

Tournament cutlines, revisited

It’s been about a month since my First look at the tournament cutlines. Since then, things have firmed up just a little bit, so it’s worth revisiting.

First a couple of things to keep in mind while looking at the pictures: 1) there are still about 250 games remaining in the regular season alone, so we should fully expect some of the “1% likelihood” events to happen; and 2) teams have wildly different numbers of games remaining in their regular season, from 6 to 12, so their potentials to make big moves will also differ accordingly.

Is anyone a lock?

Mathematically, still no. But the top four teams would need to win just one or two of their remaining games to fall out:
#1 Minnesota State
#2 North Dakota
#3 Boston University
#4 Nebraska-Omaha

#5 Minnesota-Duluth and #6 Bowling Green could each fall out with a particularly bad performance—winning about 1/3 of their remaining games.

MinnesotaDuluth

BowlingGreen

Who controls their own destiny?

Teams that should make it if they continue to do we’ll are from #7 Michigan Tech down to about #18 Merrimack, which approaches the bubble with a bit over .500 in its remaining games. Those include:
#8 Miami
#9 Denver
#10 Providence
#11 Harvard
#12 Boston College
#13 Mass.-Lowell
#14 Michigan
#15 Quinnipiac
#16 Vermont
#17 Yale

Merrimack

#19 Minnesota approaches the bubble by winning about 2/3 of its remaining regular season games.

Minnesota

#20 Colgate needs to win about 3/4 to climb to the bubble. Teams down through about #23, Western Michigan, have a similar outlook.

That includes:
#21 St. Lawrence
#22 Penn State
#23 Western Michigan

#24 Robert Morris has a tough, but mathematically possible, road to the bubble.

RobertMorris

Down through #31 Dartmouth have a similar outlook. That group includes:
#24 Robert Morris
#26 Cornell
#27 Northeastern
#29 Northern Michigan
#30 Union

Note that I left out #25 St Cloud St and #28 Bemidji St, each of which stand a slightly better (though still difficult) chance than their neighbors at climbing to the bubble.

StCloud

BemidjiState

Who needs to win their conference tournament?

Alaska

From #32 Alaska down are unlikely to make the bubble at-large, even if they win out. That group includes
#33 Clarkson
#34 Michigan State
#35 Connecticut
#36 Ohio State
#37 Notre Dame
#38 Bentley
#39 Ferris St
#40 Mercyhurst
#41 Canisius
#42 Rensselaer
#43 Maine
#44 New Hampshire
#45 Alabama-Huntsville
#46 Massachusetts
#47 Colorado College
#48 RIT
#49 Alaska-Anchorage
#50 Holy Cross
#51 Lake Superior
#52 Air Force
#53 Sacred Heart
#54 Brown
#55 Wisconsin
#56 Princeton
#57 Army
#58 American Int’l
#59 Niagara

How are last month’s predictions holding up?

Finally, let’s do a results check on last month’s predictions. The two movements that seem most surprising looking back are Harvard and Bemidji State.

I noted that no one was a lock, and that even #1 Harvard could fall to the bubble if they won only about half of their remaining games. Since then, Harvard has gone 2-6 and has fallen to #11. The current forecast matches the original pretty well—that Harvard would now need to win about 5 of its remaining 9 games to end the regular season on the bubble.

I noted that Bemidji State was the cutoff for being unlikely to advance without a major run. A 5-2-1 run since then has helped propel Bemidji State from #37 to #28. The forecasts now show that they stand a slightly better chance of making the tournament than seemed possible a month ago, but that they’d still need a run of winning at least 8 if not 9 of their remaining 10 to hit the bubble.

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

How many teams will each conference put into the playoffs?

Repeating a similar post that was inspired by message board chatter last year, I ran simulations of the remaining games and tracked how many teams each conference had in the top 14 at the end of the regular season (a reasonable guess as to the PWR rank that would guarantee an invitation to the NCAA tournament).

Let’s start with the current PWR.

Number of teams in top 14 of PWR right now
Atlantic Hockey 0
Big 10 1
ECAC 1
Hockey East 4
NCHC 5
WCHA 3

A far cry from last year when the post was inspired by inquiries about whether the NCHC was underperforming.

Now for the results of the simulations. Each chart shows the likelihoods of how many teams a conference will have in the top 14 at the end of the regular season.

aha

b10

ecac

he

nchc

wcha

Remember that the simulations assume each team will continue to perform similarly to how it has to date. So, it’s not surprising that each conference is predicted to finish with about the same number of teams in the top 14 as they have today.

More interesting is seeing how easy (or not) it is for conferences to move up or down. Atlantic Hockey is pretty unlikely to get an at-large bid. The Big Ten is more likely to fall to 0 at-large bids than climb to 2.

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

A first look at the at-large tournament cutlines

A couple of weeks ago in When to start looking at PWR (revisited), I noted that the PWR as of early January does give us some idea as to which teams might make the tournament at large. I noted that any top team can still fall out of contention, though that it takes a notable collapse for the top few. More interestingly, I observed that its unusual (though definitely possible) for a team rated much lower than 20 to climb into an at-large bid.

To see how those general historical observations will hold up this season, I ran simulations for the rest of the regular season and generated some statistics about where teams are likely to finish based on their performance over their remaining games. (Specific details about the simulations are available in the methodology section at the end).

Before we jump into the data, I want to warn that starting simulations this far out makes it pretty likely that some of the 1% events will happen. Remember that PWR cares how each team’s opponents perform, so the analysis for each team implicitly assumes that all other teams will continue to perform as they have to date. As teams’ fortunes change in the 2nd half of the season, it will affect not only their own PWR but also their opponents’. With about 450 games remaining, we should see a lot of outcomes we didn’t expect.

Is anyone a lock for the tournament?

harvard

Not completely. Even #1 Harvard could slip to the bubble if it wins only 6-7 of its remaining 14 games. That’s not particularly likely (the odd shape of the “win 6″ curve and complete absence of the “win 4″ curve are because those scenarios occurred so infrequently in the simulations).

Who can still make the tournament at-large with a good regular season performance?

northernmichigan

From #1 Harvard (as described above) down to about #27 Northern Michigan have realistic scenarios for at-large bids. It would take a good run for Northern Michigan to climb into an at-large bid; they would need at least 12 wins in their remaining 16 games to stand a good chance.

notredame

bemidjistate

From #28 Notre Dame to #37 Bemidji State, it appears possible to make the tournament at-large, but only with an amazing run (e.g. one or two losses at the most). These teams aren’t mathematically eliminated, but it’s a decent guess that being below #28 today means success in a conference tournament will be required for an NCAA tournament bid.

newhampshire

#38 New Hampshire and below look like the only path to the NCAA tournament is through the conference tournaments.

Methodology

Forecasts include the results of games played through Sunday of this week, unless otherwise noted.

Each forecast is based on at least one million monte carlo simulations of the games in the described period. For each simulation, the PairWise Ranking (PWR) is calculated and the results tallied. The probabilities presented in the forecasts are the share of simulations in which a particular outcome occurred.

The outcome of each game in each simulation is determined by random draw, with the probability of victory for each team set by their relative KRACH ratings. So, if the simulation set included a contest between team A with KRACH 300 and team B with KRACH 100, team A will win the game in very close to 75% of the simulations. I don’t simulate ties or home ice advantage.

Resources

When to start looking at PWR (revisited)

Five years ago I wrote a post for SiouxSports.com, When to start looking at PWR. I want to revisit that post because we now have five more seasons of data, including the first full season with last year’s PWR revisions.

It’s been noted countless times on message boards (by people presumably offended that others enjoy looking at PWR?) that PWR is only calculated once at the end of the conference tournaments. So why do we calculate “as if the season ended today” versions of PWR before that?

We look at PWR before the end of the season because we think it’s going to provide some insight into what that final PWR might be and what our favorite teams need to do to make the tournament. When to start looking depends what insight you’re looking for.

In this article I’ll look at how stable PWR is over time and how well it predicts the final PWR. The PWR starts containing useful information about what each team needs to do for an at-large bid as early as November. Front-runners start to become more entrenched by January. But as readers of this blog know, only the top few teams going into the conference tournaments are absolute locks, and teams as low as the mid-20s still stand a chance.

Week-to-week stability of PWR

My previous article started with a look at how stable PWR is. My thinking was that if next weekend’s games have the potential to completely upend the PWR table, then this week’s PWR table may not be particularly interesting.

PWR_weektoweekchange

The above chart shows the average PWR movement (in rank positions) of teams ranked over consecutive weeks. Consistent with the last article, PWR exhibits wild swings (an average or 4+ positions week-to-week) until the December break (movements in December are lower because teams play so many fewer games over holiday breaks). By January, movement has settled into an average of 2-3 positions per weekend for ranked teams, and down to 1-2 positions by March.

PWR’s ability to predict the final PWR

So we know when PWR stabilizes week-to-week, but what we really want to know is how good a predictor a weekly PWR is of the one true final PWR.

PWR_differencefromfinal

Though PWR seems relatively stable in January, because week-to-week movements have settled down, those movements add up enough over the weeks that January’s PWR isn’t a spectacular predictor of the final PWR. On January 1 (about 90 days before the final PWR) teams have been an average of 3-8 ranks off from their final rank. Even 30 days out teams are only within 2-4 positions on average of their final rank.

Likelihood of teams finishing in the top 12 of PWR

That’s where I stopped five years ago. Let’s go a little further—the reason we care about PWR is we want to know if a team is going to make the tournament. Let’s look at how many teams in the top 12 of PWR are still in the top 12 at tournament selection time.

PWR_shareoftop12finish

From 50%-85% of top 12 teams as of January 1 (90 days out) have finished in the top 12. At 60 days out, that has climbed to roughly 60%-85% holding onto a top 12 spot. By 30 days out, that has climbed to 75%-100%.

Also interesting is knowing how much a team’s performance to date has set their fate. Let’s look at how highly ranked you must be at given times to be a pretty good lock for the tournament and how lowly ranked you can be and still stand a chance.

PWR_highest

At 90 days out (January 1) anyone can fall out of contention, though it takes a notable collapse by a previously top-performing team. In the last ten years, we’ve never seen a team that was top 4 at 60 days out (February 1) miss finishing top 12. We’ve seen a season where the top 12 are locked at 42 days out, but also one where only 6 of the top 12 at 28 days out manage to finish top 12.

pwr_lowest

On the flip side, every year in the past 10 has seen a team ranked #17 or lower 90 days out climb into the top 12. It’s most common for a team around #20 at 90 days out to be the lowest rank from which anyone climbs to finish top 12. But, there has been a recent season in which a team unranked until Feb. 22 and ranked #25 until Mar. 15 made it to the top 12.

Effects of the new formula

It’s really too early to tell based on the empirical data if the new PWR formulas is more or less stable than the previous one. It’s a reasonable guess that the removal of the TUC cliff and introduction of sliding RPI bonuses would lead to less severe movements, but that’s not obviously the case from the available results.

I should note that the 2013 line in the first two charts isn’t directly comparable to those from earlier seasons. The 2013 PWR ranks all teams so the line represents an average of all teams, while the earlier lines only include those teams that were ranked at both times.

Some notes on statistics

Feel free to skip this paragraph if you don’t care about statistics.

I’m not a statistician and would happily take some advice from one. I didn’t calculate proper correlations in the past because I wasn’t sure what to do with the teams dropping in and out of being ranked. Giving the unranked teams the average rank of the tied group, as I’ve seen done with Spearman, struck me as potentially exaggerating those teams’ rises and declines as they fall in and out of being ranked. I suppose I could have run a standard Pearson on the teams ranked in both periods rather than just report the mean difference, but I didn’t.

But the new formula ranks every team, so without further ado here’s the Spearman rho correlation between the PWR for each week and the final PWR. Not surprisingly, you can see that even the earliest PWR rankings make a statistically significant contribution to the final PWR (with a reasonably high degree of confidence).

PWR_spearman