4/27/18 – Judging Performance From Single Data Points: Dallas Stars and Toronto Maple Leafs Edition

The Toronto Maple Leafs got bounced out of the playoffs in game seven the other night by the Boston Bruins. Everyone from Mike Babcock to Auston Matthews to Jake Gardiner is taking a lot of heat.

Gardiner sounded legitimately emotional after a tough night.

“Personally, I got to be better. A lot of this game is on me,” said Gardiner, the posterboy defenceman for risk-reward.

Gardiner’s voice quavered. Tears coated his eyes. He exhaled deeply.

“It’s just not good enough, especially in a game like this. It’s the most important game of the season, and I didn’t show up, so… there’s not much you can say really.”

Nick Kypreos got both Babcock and Matthews at the same time.

Last year, and going to this year, Matthews was the guy, and toward the end, Babcock lost Matthews. I don’t know what happened but he lost him, and there was no trust anymore. And then, Matthews can’t start a hockey game, and he can’t start a period, he can’t start a power play off a TV timeout, and for whatever reason, Babcock lost Matthews, and that played a key part. Now, he’s injured, and that of course you have to consider, but I just watched [Matthews’] body language throughout the last few games here, and going into the regular season, and you’re sitting here going, ‘That’s more than the injury; something is not right here with Matthews,’ and I think that needs to be addressed as well.

If you were on Twitter prior to the past week you are already well aware of the general feeling of unwavering almost obnoxious optimism surrounding the Leafs’ Stanley Cup chances. Fast forward to game seven and a flawed team that didn’t have to fight very hard to get into the tournament got eliminated by a superior opponent.

And the world is now ending.

The star-level player the Leafs tanked so hard to acquire is being criticized hard, some of which is fair. The highly compensated coach brought in to instill a winning culture is being raked over the coals with people just now noticing that his teams haven’t done much in the playoffs in a while. And poor Jake Gardiner who was great all season is taking a bath because of one poorly timed bad game.

Nothing about sports is really black and white. Yet, when the Leafs drop one game to the Bruins now it’s time to question everything. What happens if they had won that game? Do those questions become less important? They shouldn’t become less important if they are legitimate questions, but I think we all know how that would have turned out.

Emotion can be a powerful motivator, good or bad depending on the circumstances. It clouds our decision making ability. A burst of emotion one way or the other can lead people to conclusions they wouldn’t make if they were thinking in a rational way. This one loss, at the end of an otherwise largely successful season, doesn’t undo what the Leafs did. It doesn’t tell us much of anything about the work Babcock, Matthews, or Gardiner did over the course of the full season. It’s simply one very visible data point out of thousands of pieces of data.

The Atlantic ran a story about the impact of emotions on decision making in late 2016.

Where fear breeds uncertainty, anger instills confidence. Angry people are more likely to put the blame on individuals, rather than “society,” or fate. Anger makes people more likely to take risks and to minimize how dangerous those risks will be. Other researchers have shown that angry people rely more on stereotypes and are more eager to act.

Emphasis added.

Sometimes bad things happen. Or, to quote Auston Matthews, shit happens.

Society always tends to want someone to be held responsible whether logic dictates that anyone should be or not. We need a pound of flesh to satisfy our anger and the most visible targets are the low hanging fruit we reach up to grab.

This is all pretty standard, especially in a market like Toronto. The older I get the more it irks me. I grow to loathe the phrase “sports is a results-oriented business” and all related iterations of it more and more every year. No shit? How many businesses aren’t results-oriented?

Results are always necessary, but part of the problem is that the only results anyone wants to focus on are wins and losses. Ultimately it’s only possible to win or lose a game, but the growth and development of everyone involved is a desirable result. It just isn’t something easy to grasp so it can be more difficult to gauge.

With the Dallas Stars seeking a new head coach this problem pops up. How do you compare the win loss record of Jim Montgomery with that of Sheldon Keefe? How much credit does Montgomery get for identifying and helping Johnny Gaudreau develop? Does Keefe deserve much credit for implementing a plan with the Toronto Marlies that seems to be more or less laid out by the big club?

The same problem arises in education. In courses that culminate with a STAAR exam the results of that exam are all that really matter to many people. Allegedly that exam has all of the information needed to see how good of a job a teacher did working with a student. K. Tell me what happens when a student is absent, refuses to take it, or finishes it in 30 minutes, or has a reading disability, or doesn’t understand the references made in a question.

School districts realized that made little sense so they shifted to “growth”. Then they realized most subjects don’t have STAAR exams so some teachers had to make up their own exams used to judge their own professional growth which was predictably gamed by some. Even now growth is measured between Geometry and Algebra or from World History (10th grade) to US History (11th grade) which…ok? Good luck with that.

Any one data point is going to be very unlikely to give a full view of the performance of an individual regardless of the industry. The STAAR scores of my student won’t say much about me. This one loss doesn’t invalidate Mike Babcock or Auston Matthews or Jake Gardiner. The win loss records of Jim Montgomery and Sheldon Keefe in lesser leagues don’t automatically mean either or both will be good NHL coaches.

Bill Peters likely would have been a fine hire for the Stars despite his NHL record. Many metrics work in his favor, and he didn’t have a choice about who the Carolina Hurricanes put in net. Keefe and Montgomery have a lot going for them too. Todd Richards or any of the other number of candidates will have their own selling points to push.

However the Stars decide to fill the vacancy we won’t be able to judge the hire for several years, and even then the win loss record will be a poor way to do it. Success is more than just wins and losses and the emotional response pulled out of a loss easily clouds our view of a coach.

Good process can lead to undesirable results. If the Stars follow a good process you can’t really ask for more. As Matthews said, shit happens.



4/16/18 – Candidate Sheldon Keefe

Sheldon Keefe is the wildly successful 37 year old coach of the Toronto Marlies and formerly the Soo Greyhounds. In 500 games as a coach at the AHL and OHL levels Keefe has only lost 121 games outright.

Keefe is one of several coaches that we’ll look at who are getting labeled as “analytics guys”. We need to understand what that is and what Keefe brings as a coach to get an idea of what the Dallas Stars could be getting should they decide to pursue and ultimately hire Keefe.

The Toronto Marlies are awesome about getting videos on YouTube. They released a short video when Keefe was introduced as their new coach that gives you a brief glimpse into what he’s about now.

“I was an undersized guy that found my way into pro hockey and spending time in the NHL by working extremely hard and being relentless.”

He comes across as a very intelligent, thoughtful, and articulate guy every time I’ve seen him speak. There are tons of videos of him talking online because of the Marlies. Every one of his post game media scrums with the Marlies is on YouTube plus various other interviews.

What comes up in many of these earlier videos is his past and his connections to David Frost. I hesitate to even bring the situation up, but it’s bound to come up because of how crazy the story is, and a story as crazy as that one is bound to impact anyone involved.

If you want to read the full story, or as much of it as you’re likely to find, Gare Joyce wrote about it for ESPN. Katie Strang wrote a follow up for Deadspin.

Essentially, David Frost is a monster. The stories the reporters relay are often horrible. It boils down to an awful human being using a position of influence to take advantage of impressionable children. The story reached a crescendo when Mike Danton (nee Jefferson) tried to put an amateurish hit job on Frost.

Keefe fits into this story because he was very good friends with Jefferson. They were part of a squad known as the “Brampton Boys”. Both articles detail a little of what went on, but this one blurb from Strang’s piece is probably enough to give you the jist.

In 1996 Frost brought his Brampton Boys to Quinte, Ontario, a small town where he shacked up with several players, including Keefe and Tiveron, in “a dingy suite at the Bay View Inn.” There were reports of wild parties, puck bunny playthings, and bruised and sobbing players like Sheldon Keefe locked out of their rooms in their underwear on one of Frost’s angry whims. But the team had a rare winning season, and much was overlooked

That’s child abuse. God only knows what else went on, but speculating about that is wildly irresponsible. Needless to say, it was a bad scene that got worse over time when Jefferson ultimately tried to put a hit on Frost.

They were really good hockey players though, and Keefe was brash. He developed a reputation quickly, and he has had to repeatedly answer for it on top of all of the other stuff.

Damien Cox wrote an article about Keefe in 2015 detailing the second chance he got at hockey. One of the brash moments is so childish and immature that it’s almost laughable if it wasn’t so sad.

When they did win that OHL title, Keefe famously refused to shake the hand of league commissioner David Branch.

“This must burn your ass,” he sneered at Branch. It’s one of many things from that time he regrets.

This fight/scrum of his with now New York Islanders General Manager Garth Snow shows a little of what he was all about too.

When you carry yourself like that on and off the ice people are inevitably going to look at you crazy. Being an asshole isn’t a crime. Being an asshole and “criminal-adjacent” isn’t a good look, especially in a sport like hockey where someone like P.K. Subban gets sneered at while being a giant humanitarian.

More from Cox:

Keefe has said publicly he broke with Frost many years ago, and a phone call “would not be welcome.”


Keefe was never implicated in any crime. But his reputation was thoroughly besmirched through those relationships, and when his NHL career died after 125 games, it seemed likely we’d seen the last of him in the game.

Joyce wrote a follow up later on catching up with Keefe where he discussed the closing of those connections further.

When was the last time you had any contact with David Frost? Keefe doesn’t have a good answer to that. He’ll estimate that it’s five years, but he can’t point to a specific time or an incident…He chooses his words carefully, drip-filters all emotion from his voice. “I don’t know exactly when but I can say that he didn’t come to my wedding and doesn’t know my wife. He has never met my kids. If he called me it wouldn’t be welcomed.”

If you listen to him talk about his background at all you can tell how much it weighs on him. He touches on the Frost stuff at 10:14 of this video.

In part two of this interview he lets us in a little more.

I really wanted to have the opportunity to create an identity for myself, and if that meant having to give up playing and get involved more in the hockey operations, coaching, where again I know you would have the opportunity, in doing that, to really deal with people, young people, and their families one on one, I felt strongly that would be the best way for me to separate myself from the past and all the noise that was all around me.

He strikes me as a guy who understands the perceptions of David Frost and anyone ever involved with him, but how much can he be asked to answer for how other people acted when he was a young teenager?

The only fair question is about maturity, and he shows an uncommon amount of thoughtfulness and honesty in every answer he gives in these videos. None of this should even remotely be an issue given his resume.

And that resume is pretty damn good. Quite a bit of it does revolve around analytics. Toronto Maple Leafs Assistant General Manager, and his boss, Kyle Dubas spoke about him at the 2015 Sloan Analytics Conference.

Dubas mentioned the transformation of his team during his second year with Sault Ste. Marie. The Greyhounds were controlling about 47 percent of the shot attempts in the first 30 games and improved to 57 percent for the rest of the season.

The difference? Dubas hired a new coach, Sheldon Keefe, who took over and was open to using the data Dubas’ small team was able to deliver.

One of the slides Dubas used in his presentation is a nice little bullet point for this story. That line is when Dubas and Keefe took over the Greyhounds.

What about that means Keefe is into analytics? Nothing, but everything else written about him shows that he is. From the Cox piece again:

Heavily influenced by Dubas, formerly the Soo’s general manager, and his belief in the wave of analytical thinking sweeping over the sport, Keefe has produced a thinking man’s hockey team that relies on skill and speed less than muscle and aggression.

Given that he wasn’t that kind of player himself, the transformation is fascinating.

“I was able to put up good offensive numbers, but I was very much a straight-ahead player who relied on outworking opposition, not a guy who saw the ice particularly well,” Keefe says. “I relied on hard work, was probably over-reliant on physicality and being a pest.

“I’ve changed my outlook on the game quite a bit over the past few years.”

It wasn’t an immediate change though. As Dubas relayed to Lance Hornby.

Dubas laughed at the memory of blank looks he received from Greyhounds coach Sheldon Keefe when he first started talking about analytics. Now the team is competitive and he says Keefe is a believer.


“It took a long time to get through in Sault Ste. Marie and it’s not even close to perfect. You have to eliminate some of the noise and present the data that’s best going to help the team, whether it’s scouting, strategy or lineups. It’s trying to have everyone get on board and that takes a lot of time.

The “idea of analytics” isn’t just numbers floating into the ether, but actionable data that can make an impact. Dubas had to get Keefe to buy in, but then Keefe had to get a bunch of kids to play the way they needed to play to make the data turn in their favor.

Stories like this from Blueshirt Banter show how that process happens.

I was told a story recently where Keefe had to spend time in training camp with new players and force them to unlearn cliche concepts of getting pucks deep and making safe plays. Keefe wants a team that values possession of the puck. His teams execute breakouts so well, and do not punt the puck when under pressure in the neutral zone.

When Dubas hired him to take over the Toronto Marlies the idea was to bring those same concepts to the AHL. He wasn’t hired to focus on wins and losses, but specifically player development.

Like many clubs around the NHL, the system the Marlies and head coach Sheldon Keefe employ is virtually identical to what the Maple Leafs are doing at the NHL. But the Marlies take it a step further, structuring team meetings the same way. They attempt to replicate the environment a player will encounter should he make his way to the NHL.

“We spent a lot of time on development,” said Zach Hyman, who spent 59 games in the Marlies in 2015-16 before being promoted to the Leafs full time, and is now riding shotgun with Auston Matthews. “We have great player development people there that helped us work on things when we weren’t playing games.

“With the Marlies, they do everything that the Leafs do, and it’s an easy transition when you get called up. You know what you have to do, you know when the meetings are. You know everything. So that helps a lot.”

A few thoughts here. We’ve already seen the strong background Keefe has. We now know that he’s running the Marlies in the almost exact same manner as one of the most respected coaches in league history is running the NHL club. So if you’re big on NHL coaching experience, that seems to check that box.

More importantly, let’s take a moment to reflect how poorly the Stars have developed their prospects over the years. Rarely do you find a player joining the professional ranks with the Stars who blossoms. Development was Keefe’s primary objective with the Marlies, a team who in the last three seasons has put up two of the best seasons in AHL history.

The Leafs inevitably have a Red Wings influence to developing prospects.

“A lot of times in the NHL, we get them here before they’re ready and they get no confidence, and then we’re not happy with them,” Babcock said Thursday before the Maple Leafs defeated the New York Rangers 4-0 at Madison Square Garden. “If you get them here when they’re overripe, they’ve got a better chance of staying, and being confident players and scoring.”

As annoying as that paragraph is going to look for anyone following the Stars, it’s actually working in Toronto because they are actually developing their players because the Leafs are #ActuallyGood at it. Sheldon Keefe deserves a lot of credit for that development.

It works because from top to bottom the organization is on the same page. The Leafs draft players to fit their system who then go to the AHL and live life as Maple Leafs while being coached up by a man who has completely bought in. If you’re the Stars, don’t you want to bring a little slice of that set up into the fold here too?

The Marlies run the exact same system as the Maple Leafs. They play fast and try to keep possession of the puck. The modern game they play fits the ideas laid out by Jim Nill when he stated what he’s looking for in a coach. It doesn’t hurt that the Marlies keep staying near the top of the league in 5v5 Corsi either.

Back to that Cox story once more:

“I don’t take a lot of time to reflect, just keep pressing on,” Keefe says. “But there are times when I stop, think what I’ve gone through, think about what I’ve overcome, and been grateful for opportunities people kept giving me despite all the baggage I carried with me that would have prevented most people putting themselves out.

“Much of my motivation on a daily basis is to prove those people right.”

His strong history of player development and implementation of analytics combined with impeccable results make him a prime candidate for any opening in the NHL. He’s going to get the chance to prove those people right really soon most likely. He would be quite the bold hire for a risk averse organization, but if the Stars truly are looking for a young forward-thinking coach with a track record of success they probably aren’t finding a better candidate.