Hey folks, Phil Zito here and welcome back. In this post, we're going to start what most likely is going to be a two-part series on designing and deploying IP BAS controls. So, what we're going to tackle in this post specifically is around the designing of IP, primarily around architectures, the three-step design process, and what you need to think about when you are using IP controls. Then, in Part 2, we'll actually get into Visio, we’ll lay out some architecture diagrams, and we may run through some IP controls deployments.
IP controls, for those of you who don't know, stands for Internet Protocol. I'm going to assume that most of you reading understand what IP is. So, I'm not going to dive too deep into subnetting, IP math; I'm going to assume that's known, and I'm going to use terminology assuming that folks understand that. If you don't understand that, that doesn't mean you should stop reading. You can check out some of our Podcast episodes we did in the past here. Also, if you want to dive really deep into it, definitely check out our IT for BAS Professionals course. It's used by several OEMs to train their people on IT.
Alright, so IP controls, Internet Protocol controls, enables us to do several things. First off, it enables us to have what used to be supervisory and server level capabilities in our field controllers. Now, I don't know really what the value of having the ability to run a GUI in a fan coil controller is. There's not really one, in my opinion, nor advanced scheduling, advanced alarming, advanced trending, but you could do it now with some IP controllers. Where I really do see the benefit is by introducing IP, it's a little easier to manage and support controls, it's a little bit easier to do your downloads and uploads, it's a little bit easier to commission your devices.
Let's look at what really IP controls are, in a nutshell. So, they are a device with what's called a network interface card. This network interface card enables you to have a logical address and a physical address much like an MSTP controller, much like a LON controller. You'll have a MAC address, which is an Ethernet MAC address in this case, and you'll have an IP address which is a logical address; it doesn't physically exist.
So, you will put these IP controls in groups of networks known as subnets. Then you will connect them to, typically, a switch, a network switch . That network switch will consist of one of three architectures. We'll look graphically at the architectures in the next post, but right now I'm just going to describe them. So, a comment that is brought up is that it’s expensive. Yeah, it is, especially when you have per port costs. So, they charge you either monthly, or they just charge you permanently, per port. The reason behind that is a lot of folks think they can get a Linksys 16-port unmanaged switch for $90 on Amazon. However, when you switch over to these managed switches, whether it's Juniper, whether it's Cisco, etc, there's three types of switches.
- Type #1 is just going to be a simple layer two switch. All it does is switch Ethernet messages, Ethernet packets. The big difference here, and those costs tend to be around like $2,000-$3,000 for a 32-port switch, and the reason that's expensive is because it's a managed switch, which means you can manage each port individually. You can use things like VLANs.
- Type #2 is a layer three switch. That enables you to use IP communication, which enables you to communicate across subnets, do routing, and stuff like that. That increases the cost even more and tends to be like $4,000-$5,000 for a 32-port switch.
- Type #3 is a POE or Power over Ethernet switch, which now is even more expensive. Now you're in the $7,000-$8,000 range for a 32-port managed switch. That's Power Over Ethernet, layer three switch. So, as you can imagine, $7,000-$8,000, you divide that by 32, and you can get a pretty high cost per port, That cost per port goes down if you're leasing, but now becomes a monthly cost.
So, that's why IT departments are hesitant to give you physical ports, because they have a large cost associated with them. There are ways we mitigate that, and we'll talk about that in Part 2.
Some of the benefits I see from IP patrols as far as speed, the ability to install them, the ability to commission and program, at least for central utility plants and built-up air handlers, definitely super large benefit. I'm increasingly seeing a benefit for terminal units and fan coils and I'll cover that.
Let's start with IP architectures. So, we have three architectures:
- Star
- Bus
- Ring
Bus is super familiar to anyone who has been in the building automation space for any amount of time. We have a field bus, we have a MSTP bus, right? That field bus is just basically a daisy chain. It starts at the network interface port on the switch and it goes from device to device to device down the bus to what we formally would call end of line, but we don't have to do and have lines on a bus architecture.
Okay, now this is the lowest cost architecture. It is also the least resilient architecture. So, as far as cost goes, you only take one port, so there's a low cost there. Then you go from controller to controller, and yes, there's an increased cost for cabling and wiring, but there can be a decreased cost of operations in the ability to use faster downloads and uploads, the ability to use advanced troubleshooting tools that are specific to networking. If you are doing things that require Edge Analytics, you can do those with field bus controls. but it's going to be easier with IP based controls.
Now, ring architecture is a type of bus architecture that came to us from the industrial world. So, if you've been in industrial controls for any amount of time, you're probably familiar with industrial ring. Industrial ring control, the concept is that we are going to now have two ports at the switch. One port will be open, that'll be your primary port, and you'll have one port that is blocked. So, what happens is, all the traffic goes out the one switch port that's open on the ring, and if at any point the ring fails, like a controller goes down, a wire gets cut, then we will have the blocking port open up and it will be able to send messages. Now, why can't we have both ports open at the same time? Because we would have collisions on that ring architecture.
So, ring is the slightly higher cost because you have two end runs. You have one initiating starting end run from the switch and you have a second end run, which is your blocking side. So, you have two ports, but you have greater reliability.
Now, this is a good time to pause and talk about why, in the past, I didn't like IP architectures for terminal units and fan coil units, anything that required a non star-based architecture. The reason I used to not like this is because of what is known as electrical bridging. So, if you use an MSTP controller, master slave token passing BACnet MSTP RS-485 controller, and that controller loses power, because that field trunk is electrically bridged on the terminal block, that signal is just going to continue along to the next controller. Yeah, that one controller is not communicating because it lost power or whatever, but the rest of the controllers are.
The problem with the bus and ring architecture was that there wasn't an electrical bridge. You had two network interface cards in each controller and these network interface cards were not electrically bridged upon system failure. So, what would have to happen is the processor inside the controller would have to process the data from one network interface card and forward it out the other network interface card to continue down the bus or ring. So, as soon as you lost power on a controller, you effectively lost data transfer down the rest of the bus or ring. In the case of ring, the blocking port would open and hopefully you'd be alright.
The thing is, now most manufacturers have realized that issue and they have built in a physical electrical bridge that happens upon controller failure. Basically, the bridge happens, and it just bypasses the controller and continues down the wire. So, a lot of manufacturers have implemented that or are implementing that. So, I'm not necessarily so upset about that anymore.
Really, the added cost, in my opinion, tends to be on training your staff to be IT competent so that they can have conversations around subnetting, around VLAN setup, around route setup, etc. So, I don't know about costs outside the US, don't really keep up with that, but I will say that in my experience, the larger impact has been around specifically, IT competency, like getting your staff IT competent. It can be difficult if you don't have a structured program.
The second thing is that it also has a cost associated with IT departments and how they manage their ports. Whether they lease their ports, purchase their ports, have chargebacks for their ports, etc. So, we want to be cognizant of that.
Which brings us to the most expensive architecture, which is the star architecture. This is known as the end-run architecture. I do recommend using this architecture. A lot of folks wonder why I recommend using this architecture because it's more expensive. I recommend using it with like central utility plants, critical air handling units, critical 100% outdoor air units, etc. You're going to get a dedicated switch port.
Now I mean, yes, the switch could be a point of failure, the switch always could be a point of failure. Absolutely, but that's no different than a supervisory device being a point of failure. The nice thing is you have a dedicated run and here's where this becomes valuable. I've seen this in Europe, I haven't really seen this in the United States, although I will not be surprised if it happens. I've seen people take bus and ring architectures and plug IP cameras on to them. I've seen from certain manufacturers actually, that they talk about how their controllers have a built-in switch and you can plug in IP cameras and route them through their IP controllers, which I think is a horrible idea due to quality of service, the fact that BAS doesn't have quality of service tagging and your video stream does. That can cause basically your data to get dropped if the network gets congested.
Star is great for a central utility plant, great for those built air handling units. It's a one-to-one run so it's a single thing for you to troubleshoot if that network ever goes down. Yes, it will cost a little more in wiring and port costs, but it more than makes up for it in you having complete control about what is on that network link, and you being able to easily troubleshoot those critical systems.
Alright, so how do we design an IP control system? Like, if you were to come along a project that had IP controls on it, what would you do? One of the least talked about points in IP controls that I'm very disappointed that we, as an industry, don't talk about enough, is the fact that IT switchgear and IT assets are usually put in place in a building after Certificate of Occupancy. That's not always the case, but it is usually the case. What that means is that you have a building automation system that relies on IP controls, yet this building automation system cannot connect to the switch gear to be able to interface with one another.
So, I've seen all sorts of solutions to this. I've seen people use industrial switches to be able to put temporary switches in place in high rises or hospitals and then have IT absorb those switches. I've seen this with like the Cisco IE line. I've seen people use simple Linksys switches and connect things there. I've seen a lot of bad behavior because of this. I've seen people using /23 networks, /22 networks and building these huge subnets because they had no way to do switching, they had no way to do routing, they had no way to do VLAN segmentation. This is because, one, there was no IT infrastructure in place; and two, they had no interaction with IT because IT was disengaged from the construction process and was only really engaged after your Certificate of Occupancy. So, that is an issue that I think more people need to consider.
Three Step Process: Step 1 – Identify the Scope
The three-step process first is to identify the scope; step 1. Now what does that mean? You may understand the scope, but do you understand the networking scope? Do you understand what buildings you're going to be putting IP controls into? Do you understand where the IDF and MDF are going to be located so that you can properly end run on your communication bus? Do you have an idea of the customer's IP networking schema? Do you have an idea of how you're going to interface with customer networks? So, are you going to have to do like DMZ or going to have to do like a Dual NIC server and have the server on the production network and one port of the server on your OT network? These are all things you need to scope out and identify as you do controls, specifically IP controls.
Now I will be real with you. It's not always this complex. You know, you do a commercial office building, you can easily throw a Linksys 16-port switch in there, and probably cover a 10 story office building, pretty simply. You put a switch on maybe floor 3, a switch on maybe floor 6, and then one on floor 9. You can do 100 meters from a switch to switch, and then 100 meters from a port to device, on a star pattern. So, you should be pretty solid right?
Three Step Process: Step 2 – Locate the Network
Now the thing is, you have to locate the network, that's step 2. So, step one is to identify the scope. I want to know what devices require IP addresses. What devices am I connecting? Do I have a baseline understanding of how these are going to connect?
Then I move to locating the network. So, where's the MDF going to be? Where's the IDF going to be? Where are the switches? Where are the routers? Are we using layer three switches in the building? Is this like a collapsed core architecture? These are all questions that you should be asking yourself.
When you understand this, then you can start to figure out, this is probably going to be one subnet here, so we're going to use like a /25 or /26 cidr range on this building, and then maybe you divide it by floor, etc.
Three Step Process: Step 3 – Design the Schema
This brings us to step three, which is design the schema. So, there's a term in IT world called schema and it's usually associated with data, but schema can also be associated with networks. The network architecture, the network schema, is essentially where you're going to say, alright, I have five buildings and Building #1 is going to be VLAN-1 192.168 1.0/24. Building #2 will be .2.0/24, and so on. Then you can say that you have those, you know your BACnet network is going to be this, you’re going to have a BBMD here, and you’re going to need to have routes at this router. This is the schema you're going to design.
This is what you need to think about. This is where these things kind of start to fall apart. We’ll go more in depth in the next post, and I’ll provide visual examples.
If you have any questions, please be sure to contact us. I love answering your questions and may even do another blog around your question. Who knows?
Thanks a ton for being here and be sure to check out the next blog! Take care!
