How to Secure Your BAS - Part 3

Hey folks, Phil Zito here and welcome back. In the past two posts, we've been diving very deep into how to secure your building automation system. In this post, we are going to kind of take what we've covered so far, and we're going to add a little bit of color to it. However, we're mainly going to focus in on BAS attack methods.

So, how are people going to attack your building automation system? What controls can you put in place to mitigate those attacks?

In the previous couple posts, we went through how to establish a threat rating for your customers building, or for your building, if you're a customer watching this. We went through how you can use actuary tables from industrial attacks, and kind of figure out what is the likelihood that there is a cyber risk for my building, based on my vertical market.

Then, once you understand the attack, and how likely that is to happen, we covered some frameworks that you can implement. Specifically, we talked about the NIST 800 framework. That's kind of the preferred framework for implementing risk assessments and implementing controls.

Now, let's dive into the most common attack methods that folks are going to utilize when they are targeting building automation systems. So, the attackers have gone through the reconnaissance phase, they've decided that your building automation system or your customer’s building automation system, is an attackable asset that they want to go after. So, they're going to pursue this asset.

Well, how can they attack it? It depends on what they want to achieve. If they want to compromise the building automation system and use it as a launching point for other attacks to other systems, then you're going to see two primary attacks.

1. The phishing attack, which takes advantage of humans. This can be done over the phone, via email, via false websites, etc.
2. The malware attack.

So, there are two forms of phishing; there's bulk phishing, which is when a mass email gets sent out, and then using that mass email, hopefully someone takes action on it and gets compromised. The other is called spear phishing, where people research the individual and then make specific attacks based on that individual.

We've seen that in our organization where people have contacted some of my employees and have disguised themselves as me, saying to my employees, “Hey, I'm on a call with a customer. Can you send some gift cards to someone?” They have gone and researched me, how I communicate, and formatted their communications in a specific format that would be more likely to accomplish the attack.

So, it can be things as simple as, “Hey, can you purchase gift cards,” or “Hey, you owe taxes,” all the way to trying to get someone to click on a link and download malware. It may also be, “Hey, can you give me your username and password so I can log in and verify?” I've seen that attack actually happen more often than you'd believe, with owners who get contacted by their controls rep and their controls rep asked for their username and password because they need to check on something that's abnormal with the building automation system. So, that's how they get attacked.

Alright, so how do you deal with phishing attacks? It's primarily training. If someone is contacting you over the phone, then you'll want them to actually verify and what you can do is hang up and actually call their office back to see if that person is actually real. Additionally, if they're requesting information, you can ask people to mail you that information. You can also go to their website and actually see if they're a legitimate organization.

So, phishing is a common attack method and the only way to really counter it is to train your employees to be aware of common phishing techniques so that they can deal with them. If in doubt, escalate and verify.

Malware is the next attack vector and this one is becoming increasingly more common. There are two forms of malware, malware being malicious software. Now malware can be utilized to take advantage of vulnerabilities and control systems.

As I mentioned before, if you look at ICS-CERT, and you google your manufacturer, you will see any vulnerabilities that exist for that manufacturer software, by the version, any vulnerabilities that have been discovered. People can use those vulnerabilities a variety of different ways, encryption vulnerabilities, software vulnerabilities, authentication vulnerabilities, etc, and they can use those to compromise machines and introduce malware.

Now malware, malicious software, can be used to take advantage of the machine on which your BAS is running. Then they can use that to remotely connect in and attack business networks, or to just compromise the building automation network. Typically, I don't see people compromise the building automation network in order to take advantage of that, outside of ransomware, but you do see people compromise building automation software in order to make pivot points to business software and business networks.

You do need to be aware of this. How can you address this? Patching for one. Make sure your building automation software is patched and up to date, especially if the patch is in response to a building automation software vulnerability. So, always be checking ICS-CERT at least once a quarter, making sure your building automation software is up to date so that you are not being taken advantage of and malware is not able to be implemented.

Additionally, implement standard controls like intrusion detection, intrusion prevention, firewalls, anti-virus etc. Finally, don't download files that you don't understand, don't use the building automation server as a web browser, don't download video games or software for your personal use, etc. Those are all ways to get malicious software.

A third method that I've seen people utilize to defend against malicious software is scanning any USB device that is plugged in and sanitizing it prior to allowing it to be used. This is common with more advanced anti-virus software that it will scan USB drives, etc.

Now I want to talk about DDoS. You don't see a ton of DDoS attacks, which is a distributed denial of service attack. Basically, what happens is a bunch of people's PCs get compromised by malware, and those computers then submit ping requests or various message requests against your building automation web server, which then takes it down. The server can't keep up with the traffic requests and it crashes.

You may recall last year, there was an attack where Yahoo, and eBay, and all these people went down. It was because their domain name services servers were DDoS. So, even though if you knew the IP address of Yahoo or eBay, you could contact their servers directly because the domain name servers were brought down, the IP address could not be resolved from the domain name. So, you type in yahoo.com, the DNS server actually resolves it to an IP address so your messages can be sent. Well, since those were taken down by DDoS attacks, you couldn't connect to the servers.

So, DDoS, while not a common attack vector against building automation systems, it is one that is difficult to defend against with building automation systems, because most manufacturers are not expecting 1,000 different messages to hit their web server at the same time. That being said, if you are going to externally face your building automation service, meaning you want to make your web server externally accessible to the internet, not only should you implement firewalls, access control lists, and intrusion protection and detection systems, but you should also make sure that you're using things like proxies and traffic balancers. Cloudflare is one that comes to mind that can be used to regulate the flow of traffic to a server. It's something we implement at our company to regulate the flow of traffic to our web servers. So that way, if someone starts sending a ton of requests, the provider like Cloudflare, should be able to detect and should be able to see what is going on.

DDoS is an attack that if implemented against building automation systems, there's not a lot of counters to because people don't implement counters. Most of the counters that are implemented, most of the controls that are implemented in the building automation world, have to do with confidentiality and integrity, but not availability. So, confidentiality, making sure that the data between the web server and the server are confidential, and encrypted. Integrity, validating that the data between is not being hit by a man in the middle attack, who is, you know, changing the chiller setpoint. I mean, that's how the whole Stuxnet virus in the Iranian reactors went down. They basically made the reactors think that they were spinning slower than they were until they spun out control and destroyed themselves.

So, with all these attack methods, what can we do? Well, I'm always going to point you to NIST 853 Rev-4. If you go to NIST 853 Rev-4, you are going to see a list of security controls that you can implement. Now, the nice thing about this is there are a ton of security controls. The downside to this is that there are a ton of security controls. So, really kind of figuring out what security control you are going to implement is going to be something that you're going to make based on the assessment of risk that you decide on for your building automation system.

If you're taking a look at NIST 853, there are several different categories, and each category is associated with different things; everything from media protection to information assurance, to access control, authentication, auditing, etc. So, when you are looking, and you're trying to figure out what you're trying to protect against, you want to take that into account.

So, the thing I point out repeatedly, whenever I'm implementing controls are that there are three categories of controls.

1. Administrative controls: Training, policy, etc.
2. Technical controls: Firewalls, intrusion protection, etc.
3. Physical controls: UPS backups, isolation of your control system, locking doors, so that people can't physically access it, etc.

So, when I am thinking about those controls, and I'm thinking about, how can I implement them, I'm first going to talk through, we have to assess our risk. Now, if you are a hospital, you’re a DoD training facility, you're a data center, completely different risk profile than 3-story office building. I've said this time and time again. Once you've assessed that risk, then you need to look at your level of influence.

If you're a contractor, what level of security controls can you provide, and which level of security controls should you provide? Well, you know, if you're implementing remote access to a building automation system, you should probably implement security controls related to authentication, related to encryption, related to data integrity.

If you are simply a facility operator, and this is just like an office building, you may want to have physical backups. You may want to have a UPS backup so the system can stay online, but you did not need to implement integrity controls, or authentication controls. So, you have to take that into mind.

Now, administrative controls are related to training. As I mentioned, phishing training, training people in having an acceptable use policy on what websites you can go to, what you can access on the building automation server, what you can't access, etc. Now, realize that for almost all administrative controls, you can implement technical controls to basically enforce the administrative controls.

So, for example, if you don't want people going to certain websites, you can whitelist or blacklist websites, using your router, a data filter, a protocol filter, or a proxy server. By keeping people from accessing certain things, you can do that from a technical perspective.

This is really important, because so many people think, for example, they can’t get their people to change their passwords. Well, you can implement a technical solution to a password policy, which forces password resets, or forces complexity of passwords. So, for almost all administrative problems, outside of training, there is a technical solution.

If you're having trouble implementing password policies, you're having trouble finding out who's logging in in the middle of the night and changing set points, you can implement auditing, you can implement password policies, you can implement session timeout, you can implement session limits, how many people can log in at the same time with the same username, etc. So, there's a variety of things you can do from a technical perspective. The good thing is the majority of building automation systems should support these technical controls.

Now, physical controls, this is something that's quite often overlooked and it's kind of to my dismay, that it's overlooked. You see, when you have a building automation system, it does not cost a terrible amount of money for your main supervisory panels, and for your server, to connect to existing EM power, emergency power, and to put a UPS in place, so that you have the building automation system stay online, and then it stays online from the transition from normal power to EM power.

Something that's often forgotten, when people implement controls like that, like a physical control, like keeping the system up and running, is they may be putting UPS on the supervisory device, but not on the switch that powers the network. So, if you're going to implement a strategy like that on a supervisory device to keep it up and running, you want to do the same thing on the switch because the switch isn't going to run, then you're going to have no network traffic between supervisory devices and servers.

So, if you’re thinking through these things logically, you’d say, “Okay, what am I trying to accomplish here? I'm trying to keep my building automation system up and running from normal power to EM power switch over. Well, I can keep my supervisory devices and servers up, but if I keep them up, and the switches aren't up, they're going to stop communicating, there's going to be issues, there could potentially be interlocked logic that fails,” so you didn't achieve your objective. So, you have to think through these kinds of things.

Additionally, with physical controls, things like locking doors, access control, changing the default locks on your panels, because you have a key for a Hoffman panel, you have a key for almost all Hoffman panels. So, just be cognizant of the fact that they may have changed that recently. Last I checked, if you had one panel key, you had a panel key for pretty much all panel keys. Be aware of that and how you will protect your physical devices.

Okay, logical controls are more difficult to implement, because you're limited as to what you're building automation company and provider supports. So, do they support TLS 1.2 or above? Do they support HTTPS and certificating? Do they support auditing of their systems? Do they support LDAP so, active directory linking to usernames so that you can automatically decommission a user account if they leave? These are things you need to be asking yourself, and these are things you'll see in that NIST 853 Rev-4. You will see different controls like that.

Additionally, things that aren't thought about from a cybersecurity perspective, are things like virtualization. If you virtualize a server and you keep a server image and you keep that image off-site, then if that site is compromised with ransomware, and you have snapshots regularly being sent off site, then you potentially could have secure versions, safe clean versions of your building automation software that you can roll back to rather than pay tens to hundreds of thousands of dollars for ransomware. And ransomware is simply malware that takes advantage of a machine, locks it down, and you need to send them money, usually in the form of Bitcoin or some sort of untraceable currency, in order to get a password to unlock your machine. With ransomware, the machine is locked unless you pay the ransom, so having those versions off-site can be beneficial.

Virtualization also enables you, if a physical machine fails, to roll back right to that physical machine on a different physical machine. So, there are a lot of benefits of virtualization and creating images of your machines.

Now I want to answer a series of questions that came to us when we did our Podcast on this topic.

• Question #1: How do I recommend folks interested to try to get into the BAS industry?
  • Well, shamelessly we have had a good bit of success with our workforce development program. Not to belabor the point, but you can find out more information about that at our website.

• Question #2: How do I understand how much risk is at my site?
  • This is probably not the most difficult problem to solve. Once you know the level of risk exposure, once you know, I have 40% likelihood to have this attack hit, then you can say, “If someone hits me with a ransomware attack and takes down my building automation system for my hospital for two weeks and I was unable to run surgeries,” you can directly establish a cost against that. You can say, “The operating room makes this much revenue per day, we have 10 operating rooms, so 10 times that for two weeks times 40%, so, .4, that is our risk exposure.” Thus, you could say that comes out to be like $80,000 and we can spend up to $80,000 on cybersecurity controls, because that is our risk.

Now, this is the challenge. There is so little data on building automation attacks, that creating that risk profile, and figuring out what your likelihood of risk exposure is, is very hard. So, what I tend to tell people is take whatever vertical market you're in hospitals, airports, data centers, Google the cybersecurity risks for different attacks for those vertical markets. So, for a hospital, maybe the likelihood is 20%. Now understand that a building automation system is even less likely to get targeted, so you can reduce that even lower. So, you went from 20%, maybe to just 10%. Now, you have a building automation system that is not connected to any public network, so there's no way for it to access the internet. Now, you can reduce it even more from there. So, you can go from 10% to maybe 2% because there is that still slight possibility that someone brings in a USB drive that has ransomware on it from another compromised machine. So, then I say, “Okay, ransomware, the likelihood is 2%. How long would we be down?” Let's say, two weeks. “What would that cost us?” So, $80,000 x .02, and that gives you your cost of exposure.

From there, you can go back and say this is how much we should spend on cybersecurity controls, or training, or consulting, whatever, based on this percent. It’s not a scientific method, but unfortunately, there is just not a ton of data around building automation hacks. So, that is the method I have used in quite a few consultative engagements to analyze cyber risk.

It's a very simplified approach, but that's basically how the risk management framework works. You establish your risk, you implement your controls, you test your controls, you monitor your controls.

• Question #3: What are my thoughts on implementing BACnet Secure Connect?
  • My opinions of BACnet Secure Connect, still stand to this day. I still think it is a solution looking for a problem. I still think that there are things that are off the shelf, standard protocols that can be implemented. That being said, I do realize that there are a lot of untrained folks in the world of cybersecurity and IT. I mean, we still have an industry that by and large struggles to get IP addresses, struggles to understand routing, struggles to understand LANs and VLANs. So, embedding a secure protocol into a building automation system solution is not a bad approach. I just think we are band-aiding a problem, instead of creating a solution, which would be to recognize that our systems are primarily going IP, recognize that we're moving towards more data centric systems, and training people appropriately.

That would be the ideal solution, I do realize that that is expensive and doesn't happen overnight. So BACnet Secure Connect does seek to address that.

BACnet Secure Connect is being largely promoted as a solution for the entire building automation system. In actuality, the compromise of MSTP devices, I've never seen that actually happening. I know in theory, it can happen, but that would be a very extensive hack to pull off. You'd have to get in the ceiling, take down a controller, interrupt the MSTP bus, take that interrupted MSTP bus and then somehow form messages that exist within the BACnet framework that can be processed by a building automation device to compromise that device. So BACnet Secure Connect, I would implement it at the IP level, but I don't see the benefit of implementing it at the fieldbus level.

Additionally, you see that it is proposed that it solves the BBMD/BDT. issues. Once again, I've talked about this at length BBMDs and BDTs are issues because people don't understand routing, they don't understand broadcast domains, they don't understand basic IT concepts. They are not that difficult. I've dealt with campuses that have huge broadcast domains, and the reason that they get slow, and the reason that they are issues, has less to do with broadcast, has less to do with BBMDs and BDTs sending a bunch of traffic. They're usually running them on antiquated networks that they haven't updated, and they haven't managed their broadcast domains.

• Question #4: What level of IoT training do I need to understand cybersecurity?
  • It depends on what you If you're going to be implementing networks, which very few of you are going to be doing, then understanding things like secure access control, port access control, things like properly implementing access control lists, etc. are going to be important for you to understand. The reality remains that for the majority of you, you're still not going to be able to log into someone's Cisco switch or someone's Cisco router, or even set up someone's network security profiles for their Active Directory domains, etc.

Because of this, understanding the terminology, understanding what you can and cannot support and implement in your BAS device, those are the areas I would focus on. I would not try to learn how to implement secure tunneling on a switch or on a router or how to implement port-based access control? The reality is, most of you are not going to do that.

I would understand:

1. 1. 1. 1. What are certificates?
         2. How to create a certificate.
         3. How to go to a certificating authority and create a certificate
         4. The difference between TLS 1.0 and 1.2.
         5. What is HTTPS?
         6. What is the importance of having auditing on my building automation system?
         7. Why would I want unique usernames and passwords?
         8. Why would I want strong passwords?
         9. Why would I want auto-timeout on logged in accounts?

Those are the areas I would focus in on, and they arguably do not have huge IT requirements. Now, if you're implementing integrations, if you're implementing analytics, if you’re implementing remote access, this is where having some IT understanding would become important. The level of IT understanding that you need for our industry still remains at layer 3 and layer 4, which is the OSI model, which basically says at the IP layer, and at the MAC address layer, that's where most of our magic gets worked. That's where most of our IT knowledge needs to exist.

Now, there's definitely stuff on the database side. There's definitely stuff on the web server side, and on the server side, and that's important. Most of the time though, if you follow the manufacturer's hardening guides, you should be just fine.

Thank you so much for those of you who sent in the questions and if you still have questions, be sure to post them below. I will be sure to answer them.

Thanks a ton and take care.