Monday, August 11, 2014

Finally, hockey's great advanced stats war is over

News broke last week that the Edmonton Oilers had hired Tyler Dellow, a well-known blogger and advanced stats pioneer. The move follows on the heels of the Devils hiring Sunny Mehta, the Maple Leafs hiring Kyle Dubas as assistant general manager, and a mystery NHL team bringing aboard SB Nation’s Eric Tulsky. These days, it’s rare to go a week without hearing about yet another team staking out its territory in the world of advanced stats, or revealing that it has been there all along.

And just like that, hockey’s great Fancy Stats vs. Old School battle is over. It’s done. Soon it will seem quaint to remember that we ever argued about this stuff in the first place. There won’t be a victory parade or a formal surrender, but the numbers are here to stay, and we’re already seeing the inevitable transition from “These stats don’t tell me anything worthwhile” to “These stats don’t tell me anything I didn’t already know.” Before long the only reasons to still fight the battle will be ignorance, butt-covering, or tired shtick, and the rest of us will just move on. We won’t even hear about “advanced stats” anymore. They’ll just be stats.

It wasn’t hard to see this coming, of course, since the whole process has already played out in virtually every other major team sport. While the feud made good fodder for plenty of hot takes along the way, the end result was inevitable all along.

In recognition of the coming end of hostilities, and in an attempt to help any remaining stragglers climb onboard, let’s take one last look at some of the most common objections that have been raised to hockey’s analytics movement over the years, along with the fairly simple responses.

Advanced stats are nice, but they don’t tell the whole story

This is inarguably true. As has been well-documented, modern hockey stats are nowhere near the level of complexity and usefulness that we’ve seen in sports like baseball. There have been some significant discoveries, but there’s still a lot of work left to be done, and anyone who thinks you can replace an NHL scouting department with a spreadsheet is an idiot.

Luckily for us, those people do not exist.

That still seems to surprise some people. I’ve had conversations with more analytics folks than I can count over the last year, including most of the biggest names. I’ve followed the stats blogs. I’ve read as many of the reports as I could get my hands on, then read them again until I could start to understand them. I’ve asked questions, many of them hopelessly dumb, in an attempt to get my head around this stuff.

And in all that time, I’ve literally never once encountered anyone who thinks that stats tell the whole story. If anything, these guys tend to bend over backward to make sure you’re aware of how much they still don’t know, and how much work is left to do.

As for the common second half of this objection — “You still have to watch the games!” well, the top stats guys do. They watch more hockey than you watch. They watch more hockey than I watch, and watching hockey is my job. They watch constantly, obsessively, poring over video of long-forgotten moments, because right now that’s the only way to get the good stuff.

Stats do not replace observation. They supplement, and support, and provide a reality check for those moments when your eyes try to lie to you. There’s a balance to be struck, and as time goes on and the stats get better, that balance might shift. But it will never come close to replacing a trained set of eyes, and nobody is trying to argue otherwise.

The stat guy with all the answers and no time for old-fashioned observation is a straw man. He exists only in the realm of anti-stat fantasy. Let him go.

If advanced stats are so great, why does [advanced stat] say [star player] isn’t as good as [obviously inferior player]?

This one’s basically Mad Libs for stats haters. Let’s try a few, using last year’s numbers:

• If advanced stats are so great, why does Corsi say Jonathan Toews isn’t as good as Tyler Toffoli?

• If advanced stats are so great, why does Fenwick say Sidney Crosby isn’t as good as Andrei Loktionov?

• If advanced stats are so great, why does Corsi Relative say Drew Doughty isn’t as good as T.J. Brodie?

Those conclusions would obviously be silly, and seeing them spelled out like that could make you want to doubt whether the stats guys have any clue what they’re talking about. But none of those stats make any claim to measuring how “good” a player is. They just measure how he does in one statistic. Under the right circumstances, that information can be useful. But it doesn’t say anything definitive about who the best overall player is, and it isn’t meant to.

Russell Wilson is a better quarterback than Andy Dalton, but last year Dalton threw for more touchdowns. Does that mean touchdown passes are a broken stat? No. It just means you can’t look at them in isolation. The same is true for hockey stats, advanced or not. Goals scored is a crucial stat we’ve relied on for years, but it doesn’t tell us everything unless you want to argue that Chris Kunitz is a better player than Patrice Bergeron. Corsi, Fenwick, and friends are no different.

To some extent, this objection comes from some understandable confusion over what advanced stats claim to do. After all, there are stats like WAR in baseball that really do try to provide an overall ranking of player value. Hockey has had some attempts at a similar approach with mixed success (GVT is one example), but so far there’s no equivalent to WAR. Given how complex and free-flowing the sport is, there may never be.

No one stat is meant to definitively tell you that one player is better than another, so cherry-picking individual oddities doesn’t prove anything.

>> Read the full post on Grantland

No comments:

Post a Comment