Going Dark: A Retrospective on the North American Blackout of 2038

From March 29, 2038, to April 6, 2038, the world observed the North American Blackout of 2038. The blackout left upwards of 300 million people without power, ravaged the world economy, and devastated the global internet. By many accounts, it was the most devastating blackout ever witnessed. That said, its occurrence should not be surprising. While pundits harp on the technical sophistication of the adversary, debate the merits of a kinetic response, and politicize the blackout, the sad reality is that we have, for years, known we were susceptible to such an event. Moreover, we have had the requisite knowledge and tools to avert the blackout, but failed to use them. Plenty has been written on the wide-reaching societal effects of the blackout; our focus will be on the blackout itself. The Blackout of 2038 had two major phases. In the first phase, an active adversary exploited a vulnerability in the implementation of the Wireless Access in SCADA Environment (WASE) protocol suite that supports the grid. Grid operators acted swiftly and switched to a fallback system to restore power. Unfortunately, the adversary then subverted the fallback system by exploiting a well-known vulnerability in DNP3, a popular industrial control system protocol. The led to the second blackout. Eventually, a patched implementation of the WASE protocol suite was developed and deployed, which restored power. In hindsight, this blackout stemmed from two erroneous assumptions. First, immediately following the Texas Brownout of 2020, academics, industry professionals, regulators, and other stakeholders advocated for the adoption of a protocol that was formally verified to protect against race conditions (i.e., the cause of the brownout). However, it was wrong to equate formal verification with perfect security; we should have heeded the adage from Donald Knuth, "[b]eware of bugs in the above code; I have only proved it correct, not tried it." Second, we wrongly assumed a known-to-be-insecure fallback system would be an adequate stopgap until the primary system was back online. This paper serves as a postmortem to the North American Blackout of 2038. We analyze how the failures came to pass and the assumptions that underlie them. Moreover, we offer a complete and simple solution to prevent these conditions from ever arising again: the adoption of Language-theoretic Security (LangSec) principles. To this end, we provide and evaluate a preliminary implementation of a LangSec parser for the WASE Short Message Protocol format (WSMP). Additionally, we urge lawmakers and regulatory agencies to mandate the verification of fallback protocols.