IN A famous detective story by Edgar Allan Poe, The Purloined Letter, a minister steals a letter containing compromising information from a woman and uses it to blackmail her. The police scour every corner of his hotel room in search of the document: they check behind the wallpaper, under the carpets, and even examine the tables and chairs with microscopes, all to no avail. Defeated, they summon C. Auguste Dupin, an amateur detective, to help them with the case. Mr Dupin surmises that the minister would try to outwit the police by leaving the letter in plain sight. He duly spots it tucked into a card rack, delivers it to the authorities, and collects a reward of 50,000 francs.
According to a series of news reports, the C. Auguste Dupin of the National Basketball Association (NBA) this year is Ed Weiland, a driver for Federal Express in Bend, Oregon. The personnel experts of all 30 NBA teams poke, prod, and spend millions of dollars attempting to find the best players in the leagues annual draft of amateur players. But in 2010 not one of them was able to identify Jeremy Lin of Harvard as a future professional star. He went undrafted, and was later ignored as a free agent by virtually every NBA team.
Meanwhile, Mr Weiland, who contributes to the HoopsAnalyst.com website, singled out Mr Lin from among the 5,000 or so players at Division I colleges, and wrote that his statistics suggested he had a good chance to succeed in the NBA. Once Linsanity erupted on both sides of the Pacific following Mr Lin’s incandescent start for the New York Knicks, Mr Weiland became a minor celebrity. The story of the hidden genius giving the experts their comeuppance is just as compelling today as it was when Poe wrote it in 1844right?
Not so fast. It is true that Mr Lin has turned out to be a very good player. But Mr Weilands seemingly clairvoyant forecast is a red herring. In fact, no one could have predicted the level of play Mr Lin has attained at least not without mistakenly foreseeing similar achievements for dozens of other players as well. Rather than life imitating Poes art, what Mr Lin’s story really demonstrates is the old Niels Bohr adage: prediction is difficult, especially about the future.
As in many other professional sports, the basketball world today can be divided between a new wave of objective statistical techniques and traditional methods of visual observation. Both schools of thought, however, rely on historical examples in order to produce their predictions.
Looking to the past is indisputably the best way to shift the odds in a forecasters favour. However, in order for it to be successful, the historical cases it is based on need to be reasonably comparable to the case in question. You would be rightly sceptical of someone who attempted to predict height from shoe size for women based on a sample of men.
Similarly, Mr Lin’s story is extremely unusual. There have been precious few point guards in the NBA who, like Mr Lin, were not recruited by college basketball teams while in high school, played against weak competition while in college, and played a position other than point guard for most of their college careers. When comparables are hard to come by, the flaws in a historically-based method are laid bare.
Defenders of the statistical approach will insist that there was ample quantitative evidence of Mr Lin’s potential. As Mr Weiland noted, he was a dominant guard at Harvard. The models used by teams and amateur analysts alike to predict how college players will fare as professionals invariably identified him as a strong NBA player.
However, those models suffer from an inescapable selection bias: they are based only on the roughly 2% of college players who went on to play in the NBA. As a result, they are all but useless when applied to the broader pool of all college players. In the 2010 draft alone, there were more than 20 other players who had faced relatively weak competition whose statistical probability of NBA success was similar to Mr Lin’s.
Take, say, Josh Slater, a guard from Lipscomb University who is the same height (six feet, three inches or 1.91 meters) as Mr Lin. Mr Slater averaged 17 points, five rebounds, five assists, and two steals per game, a dead ringer for Mr Lin’s 16 points, four rebounds, five assists, and two steals per game. Just as Mr Lin put up 30 points, nine rebounds and three assists against Connecticut, a powerhouse team, Mr Slater posted 21 points, nine rebounds and four assists versus North Carolina, another elite basketball school. Was Mr Slater, and all the other college players like him, unjustly passed over as well? Could your favourite NBA team go on a magical playoff run by inserting him in their starting lineup?
Of course not, counter advocates of the traditionalist camp. Mr Lin and Mr Slater might look the same in a spreadsheet, but they couldn’t look more different on the court. All one had to do, they claim, was watch Mr Lin play, and shame on those teams like the Golden State Warriors, Dallas Mavericks and my own Houston Rockets, who had him in their organisation and failed to recognise his brilliance.
Neither argument is completely wrong. Maybe Mr Slater could succeed in the NBA, and maybe Mr Lin should have been given playing time sooner. But the structure of professional basketball makes it impossible for teams to give a chance to every prospect who shows some potential.
For example, Major League Baseball (MLB) drafts some 1,500 players a year. Once selected, they must work their way up four levels of minor-league teams before joining the parent club. This gives MLB organisations the incentives of salmon, which spawn scores of young far upriver in hopes that a handful make the treacherous journey all the way back to sea. Each team can invest in hundreds of prospects and see which pan out, needing only a few winners that hit it big to make up for all the failures.
In contrast, NBA teams cannot hold the rights to anyone beyond the 15 players on their active roster. That makes them more like elephant mothers, who give birth to very few babies and have to gestate them for almost two years. With limited investment opportunities, teams are forced to choose only the players with the greatest likelihood of success, and then give them a long-term contract and a potential path to significant playing time. And no one could have called Mr Lin a high-probability prospect. In the end, he only got his break after he had polished his game for a year in the NBA Development League (D-League), basketballs minor league, and when a series of injuries on the Knicks created an opening at his position.
The NBA has made some movement towards a model more like baseballs. It launched the D-League a decade ago, and is now trying to promote exclusive relationships between NBA and D-League clubs. In fact, in 2009 the Rockets became the first NBA team to pair up formally with a D-League counterpart in a baseball-like hybrid-owner relationship, when we entered a partnership with the Rio Grande Valley Vipers. Other NBA clubs, such as the San Antonio Spurs, have chosen outright ownership of a D-League team. Hopefully, in the long run, this will allow teams to provide opportunities to more prospects than they can now. Nonetheless, the elephant-mother model will remain entrenched for at least the next few years, since NBA teams can still hold a maximum of 15 players, regardless of whether they are affiliated with a D-League club or not.
Mr Lin has received so much attention because he embodies the reason we love sports: every time you watch, something amazing might happen that no one anticipates. He is an outlier and an underdog whose hard work is paying off at last. Just don’t tell me that anyone, even C. Auguste Dupin, could have predicted it.
NOW WATCH: Briefing videos
Business Insider Emails & Alerts
Site highlights each day to your inbox.