After years of hype about its autonomous driving system, the facts are coming out about how safe Tesla and Autopilot really are.
Elon Musk’s company admits three of its vehicles have crashed while Autopilot was engaged, including one fatal accident in which Joshua Brown’s vehicle ran directly into a semi truck. In the wake of Brown’s death, Musk claimed Autopilot would save thousands of lives if it was deployed universally today. That bold claim is based on insufficient data and flawed comparisons between Tesla vehicles and all other cars that stack the deck in favor of the company.
Musk’s fame as a self-described “applied physicist” and serial entrepreneur have generated a seemingly inexhaustible public faith in his intelligence and leadership, but by promoting such a flawed statistical comparison of his firm’s controversial system he calls that confidence into question.
ADVERTISEMENT
Autonomous drive technology was never mentioned in Musk’s ambitious 2006 “Top Secret Master Plan” for Tesla, and appears to have been tacked on to its world-changing mission in response to Google’s massively-hyped (but still not yet commercially deployed) self-driving car program. The reveal of Google’s radical, zero-human-control autonomous concept car in 2014 suddenly made Tesla’s cars-of-the-future look decidedly passé. And in contrast to Google, whose communication about autonomous drive focused entirely on the technology’s long-term safety benefits, Musk’s fixation on beating the competition to market looks more like a rush to protect Tesla’s high-tech image than the pure pursuit of safer roads. When Morgan Stanley in 2015 boosted Tesla’s stock price target by 90 percent based on the projection that Musk would lead the auto industry into an “autonomous utopia,” it showed that even a perceived advantage in autonomous drive could help Tesla raise the huge amounts of capital it needs to continue growing.
Musk set about creating this perception in 2013, when he said that Autopilot would be capable of handling “90 percent of miles driven” by 2016. By mid-2014, Musk was promising investors that the system could handle all freeway driving, “from onramp to exit,” within 12 months of deployment. Tesla still has yet to make good on those bold claims, and competitors argue that Tesla’s relatively simple sensor hardware will never be capable of safely performing at such a high level of autonomy.
In the wake of Brown’s fatal crash, Tesla’s sensor supplier Mobileye clarified that its current technology is not designed to prevent a crash with laterally moving traffic like the turning semi truck Brown’s Model S struck. This week, Tesla revealed another Autopilot accident that saw a Model X swerve into wooden stakes going 55 mph in a canyon road.
Experts have understood Autopilot’s hardware limitations for some time, but Tesla owners and investors clearly believed that Autopilot was either an autonomous drive system, or something very close to it. Brown clearly believed that Autopilot was “autonomous” and described it as such in the description of a video that Musk shared on Twitter. So great was his apparent faith in Autopilot’s autonomous capabilities that he was reportedly watching a DVD at the time of his fatal crash. The extent of Autopilot’s true abilities, which wax and wane with each Over The Air software and firmware update Tesla pushes to the car, is hotly debated on Tesla forums where even Musk’s most devout acolytes waver between extolling its miraculous powers and blaming drivers for their inattentiveness depending on the circumstances.
This ambiguity and overconfidence in semi-autonomous systems is why Google refuses to develop anything less than fully autonomous systems which requires no driver input, a level of performance the search giant insists requires the extensive testing and expensive LIDAR sensors that Musk has often dismissed. It’s also why major automakers are developing driver alertness monitoring systems that they say will keep drivers in semi-autonomous vehicles from relying too heavily on their vehicle’s limited capabilities.
Rather than waiting for LIDAR costs to come down or building in a complex driver alertness monitoring system, Tesla has chosen to blame its faithful beta testers for any problems that pop up in testing. One Tesla owner describes this Catch-22, after being told that a crash was her fault because she turned off Autopilot by hitting the brakes: “So if you don’t brake, it’s your fault because you weren’t paying attention,” she told The Wall Street Journal. “And if you do brake, it’s your fault because you were driving.”
Confusion about Autopilot’s actual abilities has persisted even after the first fatal crash was reported, with Musk dismissing questions about the wreck by claiming that road fatalities would be cut in half “if the Tesla Autopilot was universally available.” The shaky statistical basis for these claims is just the latest in a long line of confusing and contradictory statements about Autopilot’s abilities.
Musk first claimed that Autopilot is twice as safe as a human driver before any reported crashes involving Autopilot, when he asserted that the average distance driven before an airbag deployment was twice as long for vehicles with Autopilot. This position ignores the fact that airbag deployments are not the same as fatalities or injuries. In fact, about 3,400 Americans (about 10 percent of total annual road deaths) die each year in frontal crashes where airbags are not deployed. Moreover, Musk was drawing on just 47 million miles driven in Tesla vehicles, or less than 0.00001 percent of the more than 3 trillion miles driven by Americans in 2014.
The fact that Autopilot is only supposed to be activated on divided freeways, without cross-traffic, cyclists, or pedestrians skews the statistics so far in its favor that any comparison with broader traffic safety statistics “has no meaning” according to Princeton automotive engineering professor Alan Kornhauser. And since Tesla has blamed drivers for wrecks in which they turned off Autopilot features by attempting to steer or brake just before a crash may also be limiting the number of incidents Tesla reports as involving Autopilot.
In Tesla’s response to the recent fatality, the company emphasized that Autopilot is responsible for fewer fatalities (one per 130 million miles driven) than the overall U.S. fleet average (one per 94 million miles driven). The accuracy of the latter figure has been called into question by Sam Abuelsamid, who points out that vehicle occupant deaths in the U.S. occur only once every 135.8 million miles, making the average U.S. fleet slightly safer on average than Tesla’s one death per 130 million miles.
In addition to being potentially inaccurate, the company’s statistics are also fundamentally lacking in comparability because they fail to account for significant differences in vehicle age and vehicle cost—two attributes that significantly affect vehicle safety.
Tesla compares its fleet of (on average) 2-year-old cars to the U.S. fleet and its average age of 11.5 years, almost as old as the automaker itself. Data from the Insurance Institute for Highway Safety show that even when vehicles from 2004 were new, they were much less safe than modern vehicles are. At the time of manufacture, these vehicles were responsible for 79 fatalities per million registered-vehicle years—by comparison, the 2011 model-year vehicles cut the fatality rate by nearly two-thirds, to just 28 per million vehicle years. This yawning gap in safety is even wider now, since the 2004 vehicles are now over a decade old, and have likely racked up over 100,000 miles. Tesla holds up this aging U.S. fleet (including motorcycles, which are 26 times as likely to be involved in a fatal accident as passenger vehicles) as a reasonable safety comparison for its fleet.
Tesla’s comparison also overlooks the dramatic cost difference between its luxury cars and an “average” vehicle on the road in the U.S. today.
The lowest-priced Model S starts at $66,000, about the same as an entry-level Audi A7, and climbs to around $135,000 with options, a similar price point as a fully-optioned Lexus LS 600h hybrid. The Model X starts around $83,000 and tops out around $145,000, a price range bracketed by the Porsche Cayenne SUV on the low end and a Porsche 911 Turbo on the high end. Vehicles in this price range are engineered to be some of the safest vehicles on the road, and many come with a panoply of advanced driver assistance safety functions, like Automatic Emergency Braking and Adaptive Cruise Control, making them the most direct competition for Tesla’s vehicles. Comparing a vehicle in this rarified segment to the overall market, where the average new vehicle costs less than $35,000, is beyond disingenuous.
Given Tesla’s sophistication and resources—and its strong incentive to make a robust case for the safety of its vehicles—it is surprising that they weren’t able to put together a more compelling comparison.
If federal safety regulators find that Musk’s public statements about Autopilot’s abilities similarly misleading, his bold “public beta test” could set back more than just his image and Tesla’s; it could raise suspicions that compromise the development of autonomous drive technology more broadly.
