Seven minutes before midnight last Dec. 17, a bomb of sorts went off in a high-voltage substation north of Kiev.
But if you were standing outside the 20 acres of gleaming metal transformers and coils, you wouldn’t have heard a bang or seen a flash. It wasn’t that kind of bomb. It was a piece of malicious software that had been hiding in a control-room computer miles away, waiting for the right time to reveal itself. At 11:53 p.m., the logic bomb transmitted a staccato burst of pre-programmed commands to the substation, popping one circuit breaker after another until a strip of houses in and around western Kiev were plunged into darkness.
Technicians responded to the Pivnichna substation and took the circuit breakers off computer control, restoring power a little after 1 a.m. It was only the second confirmed case of a computer attack triggering an electrical blackout, and compared to the first, 12 months earlier—also in Ukraine—it was a fizzle, affecting far fewer customers and for a fraction of the time. In the six months since the Kiev attack, security researchers have wondered why the hackers even bothered with such a fleeting disruption and speculated that someone was using Ukraine as a testing ground for a more serious attack.
ADVERTISEMENT
Now that dark assessment seems to be confirmed. Researchers at two security companies on Monday announced they’ve finally found and analyzed the malware that triggered the Kiev blackout, and it’s far worse than imagined. The computer code, dubbed “CrashOverride” by Maryland-based Dragos, and “Industroyer” by ESET in Slovakia, is a genuine cyber weapon that can map out a power station’s control network and, with minimal human guidance, issue malicious commands directly to critical equipment. Only once before has the world seen malware designed for such sabotage, with the 2010 Stuxnet virus used against Iran’s nuclear program. CrashOverride is the first to target civilians and the first such malware built to target a nation’s power supply.
It’s unclear who created CrashOverrride. Both ESET and Dragos say it was built from scratch, leaving none of the usual fingerprints that allow analysts to link one hacking campaign to another. Ukraine has faced a near-biblical plague of cyberattacks since entering into hostilities with Russia three years ago, and many have led unequivocally to Moscow. But not so with CrashOverride.
The only thing that’s certain, says security researcher Robert Lee, CEO of Dragos, is that the malware wasn’t built as a one-time weapon. It’s designed from the ground up to be easily reconfigured for a variety of targets and contains some payloads that weren’t even fired off in the Kiev attack.
“It’s a nightmare,” Lee said. “The malware in its current state would be usable for every power plant in Europe. This is a framework designed to target other places.”
ESET was first to find samples of the malware, and the company shared its initial analysis with Dragos, which went on to find additional samples and new components of the code. Electric utilities throughout the United States and Canada were alerted to the new malware last week by the North American Electric Reliability Corp., the industry group responsible for power-grid security.
“We believe that our current protective measures provide an initial barrier,” said Marcus Sachs, NERC’s chief security officer, “and we are providing additional technical information to North American utilities specific to this malware.”
CrashOverride marks a significant escalation in the electronic arms race, at a time of overt saber cyber-rattling from U.S. adversaries like Russia and North Korea, and increasingly loud warnings about the vulnerability of the power grid. Last January, the Department of Energy assessed that the U.S. now faces “imminent danger” of a cyberattack that would trigger a prolonged cascading outage that would “undermine U.S. lifeline networks, critical defense infrastructure, and much of the economy; it could also endanger the health and safety of millions of citizens.”
Lee says CrashOverride is built to cause regional outages and in its current form doesn’t have the capability to start a cascade on the order of the 2003 Northeast U.S. blackout, nor to be easily repurposed to target other industrial control systems like water-treatment plants or gas pipelines. But the amount of expertise and resources that went into creating the program augurs even more dangerous malware to come. “What makes this thing a holy-crap moment is the understanding of grid operations encoded within it,” he said.
That’s because the code targets a crucial technology called SCADA, for Supervisory Control and Data Acquisition. A SCADA network is essentially a electronic nervous system that allows operators to remotely monitor and control all the pumps, motors, relays, and valves that undergird society’s infrastructure. The technology grew out of the electric industry beginning in the 1940s as a solution to the growing complexity of power distribution, which requires constant monitoring and adjustment of equipment at thousands of substations scattered around the country. Rather than keep technicians at every site, utilities began connecting the substation equipment to meters and knobs at centralized control centers, first by wire, later by radio, and today over serial ports and digital networks, with graphical computer controls replacing the meters and knobs.
As products of a more innocent time, the major SCADA protocols were never designed for security. “We use the term ‘insecure by design,’” said veteran SCADA security guru Dale Peterson. “You can switch relays on and off without any authentication. Everything an attacker would want is a documented feature of the device.”
By the 1990s, the U.S. was eyeing SCADA as a potentially critical vulnerability. In 1997, President Bill Clinton ordered a risk assessment of the power grid, and his advisers found it riddled with holes, including equipment reachable through corporate networks and open dial-up modems.
The electric industry has been developing and enforcing stricter security standards ever since. But with the entry of nation-state cyberattackers the risks have only grown, and the industry now regards cyber blackouts as something to plan for, like the inevitable outages triggered by extreme weather. The key, said NERC’s Sachs, is to “ensure rapid restoration should an outage occur, regardless of the cause.”
That gloomy outlook owes much to the first Ukrainian power hack in December 2015. In that attack, intruders used conventional hacking tools and techniques to seize the Windows machines in a utility control room, where they dragged the mouse cursor across the screen and clicked on the controls for a trio of local substations. The resulting blackout left 225,000 people without power. Ukrainian security services attributed the attack to Russia.
While successful, that attack suffered from a critical weakness from a cyberwarfare perspective: It didn’t scale. The hack required manual effort by a control-system expert sitting at a keyboard. That limitation is obliterated by CrashOverride, which, once it is configured and deployed, operates invisibly and automatically at the lowest levels of a plant network.
The code used in Kiev included swappable modules for four SCADA protocols prevalent in Europe. When the proper module is activated, it runs under the name of the legitimate Windows process controlling equipment at the remote substation. CrashOverride kills the original program and starts issuing its own commands over the SCADA link, cycling through a range of circuit-breaker addresses and systematically tripping each of them, then starting again at the top. Even if the control center is able to send its own commands to restore the circuit, CrashOverride will just hit the breaker again, running continuously in an infinite loop. “It’s like a popup on a website where you close it, and it just keeps opening again,” said Lee. “That’s what they’re doing to circuit breakers.”
Peterson said he expects CrashOverride to inspire copycat efforts, particularly among nation-state attackers. “To see something that’s been predicted for so long actually happen… More people will think they should be doing it.”
“If we haven’t had enough of a wakeup call already, this is it,” said Dragos’ Joe Slowik, who helped analyze the code. “The time is running out until someone either gets lucky or deliberately targets a network that all U.S. citizens care about, instead of saying, ‘Oh, it’s Ukraine. Who cares?’”
