Bleeding Hearts

So much can change in a week. A scant few days ago, people believed that seeing that little padlock on a website's address bar meant that their information was secure. They believed that networking hardware would protect their users from intruders. They believed their phones and other devices were relatively secure.

The discovery of the Heartbleed bug has turned everything on its ear.

Heartbleed, a programming error in OpenSSL, a popular implementation of the SSL security standard, allows an attacker to capture user IDs, passwords, encryption keys, and content, completely undetected. The frightening thing is that it's actually been in the code since December 2011, although it was only discovered by researchers this week. Because attacks enabled by the bug are undetectable, it's impossible to tell whether cyber-criminals discovered it earlier, and have been merrily scooping information all along. Rumours are now emerging (since denied) that the US National Security Agency (NSA) has been using the flaw in its activities for two years; do we believe crooks are far behind?

OpenSSL is used by many of the Web's most popular sites, including Yahoo!, Dropbox, Blogger, Google, Flickr, GoDaddy, NetFlix and Facebook. It is an open source implementation of SSL that is used, according to Internet infrastructure analyst firm Netcraft, in two-thirds of active websites. The bug, the result of a tiny programming error, is having huge consequences, undermining the entire security structure of the Internet and of corporate networks.

To remediate the problem, affected websites need to first patch OpenSSL, then revoke and replace their sites' digital certificates before they're again secure. But despite the reported flurry of patching, Netcraft says that only 30,000 of the 500,000-plus affected digital certificates, which provide the cryptographic keys for each site, have been re-issued, and even fewer of the potentially compromised ones have been revoked. Un-revoked certificates allow criminals who have access to them to impersonate their rightful owners. Furthermore, Netcraft has noticed errors in some of the re-issued certificates, probably due to the scramble to get them out as fast as possible.

With hardware devices, the problem is even messier. Companies like Cisco and Juniper Networks have already admitted that some of their equipment contains flawed versions of OpenSSL. With them, we have to wait for the vendors to develop and test device-specific patches, then apply them. That will involve outages companies can ill-afford, and continued risk while the patches are being developed.

After patching and certificate revocation and replacement is complete, users still need to change passwords for affected websites and devices (any previously extracted information will still be in criminal hands, however). But they should wait until notified that a site or device is secure, so the new passwords can't also be compromised.

The whole mess illustrates our dependence on that fragile entity known as open source, and the risks attached thereto. Yes, it's free, or cheap. Yes, in theory, many eyes on the code should mean fewer issues, and quicker discovery of issues that do sneak through. In theory.

However, the error that caused the Heartbleed vulnerability is a "programming 101" type error — the developer forgot to tell the program to check that the length of information input fit into its allocated space. If data exceeds the space, it slops over into memory destined for other purposes, and can cause unexpected results. A novice’s error, yet both the developer, and the person who checked the code, missed it. Let’s be charitable and just call it sloppy work, or the result of poor training, but the consequences remain.

Bottom line: if we’re to rely on open source for critical components of infrastructure, programmers need better training, and those who check their work must be more diligent. That means funding for the projects. Open source is NOT free when we measure the cost of recovering from errors like Heartbleed!