That AMX Backdoor

netlinx-controller-0116The AV industry received some unsettling news last week regarding a potential security vulnerability in Harman’s control processors. It was a rare moment when we as a group were all in the news, though not in a good way. For those who’ve not seen it by now, Ars Technica reported that AMX (now Harman) Netlinx processors have a “back door” which could, in theory, create vulnerabilities to certain kinds of outside attacks. Two questions come to my mind: How worried should we be, and how can we do better in the future?

First, to clarify, the back door in this case was took the form of secret user-account and password that could be created by executing an undocumented command via telnet or SSH. This account had not only full access to the device but, as a diagnostic tool, could be used as a packet-sniffer. Was this dangerous?

At the very worst, yes. An attacker could log into the device, view some network traffic (though only that traffic going to the port on said device) and, if they also knew how to read AMX code and figure out how the system was constructed, turn on microphones, cameras and even secretly dial a video or audio call during what should be a private meeting. That is the worst-case scenario.

Now that we’re all scared, we can ask: How likely this is? In my estimation, not very. The first reason is reminiscent of the vulnerability discovered in Nest learning thermostats two years back; yes, an attacker can access the device in ways of which you wouldn’t approve. This requires, in the case of the Nest, physical access to the device. In the case of the Netlinx controller, one would need access to the network on which it resides. AV control networks are almost always segregated to their own network, either physically or logically separated from the larger converged network. Best IT security practices would require some form of authentication (through a dedicated VLAN or similar) before accessing such a network from the outside. So, in other words, if an attacker can access a back door such as the AMX “Black Widow” account (more on the name later), there has already been a failure of network security. It may not be as obvious as the attacker having broken into your home to access your thermostat, but it is a breach nonetheless.

There are two bigger picture pieces here. One is that we DO need to be serious about this kind of vulnerability, for the sake of perception if nothing else. Where Harman failed was not primarily in leaving a potential backdoor open, but by not answering a security consultant for several months until the weakness was made public, forcing them to react to an uncomfortable story. Bigger picture yet, it’s no secret that AV and IT are increasingly converging and that we are living on larger converged networks. IT is a much bigger world than AV and, in many ways, a far more mature industry when it comes to concerns about security. In other words, we’re sitting at the adult table and we need to act like it.

The first step in acting like adults is to recognize that security is a serious issue and — whether as manufacturers, consultants or integrators — be mindful of our clients’ safety and privacy. Yes, that means that AMX (and other control manufacturers out there) shouldn’t leave back doors, no matter how much easier that might make later troubleshooting and support. It also means that on installation we shouldn’t leave the front door open. How many installed processors are out in the wild with the admin passwords still set to the factory default? I’m certain that it isn’t all of them, but I suspect it’s far more than it should be. One suggestion I’ve seen (ironically, from Harman’s Paul Zielie in a presentation about security) is to reset all of the admin passwords to something project-specific rather than generic prior to shipment and installation. This is far from perfect (especially if ones idea of “project specific” is the street address to which it is being delivered), but is a tremendous improvement over a factory-default, which anyone can look up. We need to not just assume that AV devices will be segregated to their own network, but include language in our specifications and instructions to IT managers that a network-based AV control device can be as much a security risk as any other network appliance.

See also  Notes From the Trenches

The second step in acting like adults is to act like adults. There is one thing I am now requesting from all of you programmers, manufacturers and contractors out there. Yes, you. Do your work as if it’s going to end up in a story in Forbes, because it quite possibly is. I can’t ask to never again read an article about something someone in the industry did wrong; we strive for perfection but need to be reasonable enough to not expect it every time. What I can expect is to never again read an article about a vulnerability or failure in an AV system and read a phrase in a major news source such as “a leet speak version of ‘I’m Batman.'” We’re at the adult table now. There are people in the AV industry who have, in their reactions, defended the sense of humor of the programmers at Harman who chose to use comic book names for these features and secret accounts. I believe in humor as much as anyone, but those of you who have followed me over the years knows there’s something in which I believe even more.

I believe in narrative.
I believe in metaphor.
I believe in stories.

When the world learns that a back-door was left in an AV product and that the back-door was named after a comic book character and THEN (and I’m taking Harman at the word on this next part) a similar feature is shown to ALSO be named after a different comic book character — that tells a story. It says that we’re not taking ourselves or our work seriously. It says that we’re overgrown nerd-kids whose products don’t belong in Fortune 500 companies, in government facilities, in any places which take their work seriously.

It says that we’re kids sitting at the adult table.

The sin in all of this is that we are not children. Every AV manufacturer with whom I’ve discussed this takes security seriously. Harman, as I said, has their representative educating the industry through an InfoComm course. Crestron has, wisely, chosen to not only move to fully standards-based communications and security protocols, but to remove default passwords and set more secure modes of operation as their factory default settings. We are, as a group, learning — perhaps more so than they myriad newcomers to the network space coming as part of the “Internet of Things.”

None of our gains matters in terms of perception so long as we not only make mistakes but make them in memorable, splashy, and unprofessional-looking ways.

So please, be careful. Be thorough. Be mindful.

And be mature. Save the in-jokes for the office holiday party. I’ll thank you for it.

Editor’s Note: AMX released a statement regarding the vulnerability that said there had been no actual security breaches of which it was aware, and that a firmware update was deployed in December that disabled the access, which the company also said was originally intended for assisting in remote diagnostics and troubleshooting.