Hey folks, Phil Zito here and welcome back. In this post, we are going to be continuing along with talking about deploying IP controls and designing IP controls. So, in the previous post, we went through IP controls frameworks, things like that. Now we're going to start with some relatively simple diagrams, just talking about parts and pieces, and then we're going to get into some more complex diagrams and general theory.
Alright, so just a brief recap of the last post where we talked about IP controls and what they are. We talked about there being three architectures. There's going to be a ring architecture, a star architecture, and a bus architecture. We talked about when you would use each one and we talked about the pros and cons of each one. We talked about the basic process for initiating an IP design. That is step #1, identify the scope, step #2, locate the network, and step #3, design the schema.
In this post, we're going to start off with a really simple star architecture. We're going to move from there to a bus architecture. We'll move from there to a ring architecture. Then we'll start looking at multi building architectures and we'll talk about concepts like routes, VLANs, etc.
So, the first concept we're going to talk about is the basic star pattern. This is what most of you are used to when you're setting up supervisory devices and some of your IP controllers at the plant level. So, we start with a switch, and as we recall from the last post, this is typically a layer two switch, meaning that in order for you to get to any other part of the network, you have to use routers, if you're stepping between subnets, but we're not going to be doing that just yet.
We're assuming that we have a supervisory device on one physical port. For the sake of these diagrams, supervisory devices are going to be a box with an S, controllers are going to be a box with a C, and then servers will be a box with like a keyboard layout on it. So, we have a supervisory device, and this is a star run right here. So, right here, this white line from the switch to the supervisory device is a star run.
Now on the switch, we'll have another port and this may go to a controller. This would be a central utility plant controller, AHU controller, maybe even a field controller.
Now, we have maybe this, and on this controller, if we're using some manufacturer’s controller software or their supervisory devices, they may have a field bus. So, our field bus would go to our controllers, this would be something like an MSTP bus and we would have daisy chained controllers with our typical EOL at the end of the run.
So, this is a basic star pattern. It's what most of you right now are using across most of your jobs. Not very confusing, pretty straightforward. We get a design sent to us, and it shows a two-story office building, maybe it has an AHU and a dozen VAV boxes. So, maybe this is what we do, we have a supervisory device, a controller, and we have on our trunk, some field controllers.
Now we can make this with newer systems and newer architectures. We can actually make some changes to things here. What we can do with some of these newer architectures, and newer controller lines, are that some of the plant level IP controllers are actually coming with MSTP buses on the controller. So, what I'm now able to do is have my controller, which will have my graphics, it will have my trending at a low level, alarms, etc. Then from here, I will have my trunk, so I'm able to reduce that supervisory device costs.
So right here, I'm making a shift with IP architecture where I'm still using a hybrid of MSTP and IP, but you can see I'm doing that and then my air handler would be on this controller right here.
Now we're going to go to a slightly more complex architecture, still single building, still single subnet, but it is going to be a bus architecture. Now, what we may find ourselves doing is for whatever reason, we may find ourselves being expected to do an all-IP architecture. What we would find is same building, we have an AHU, and let’s just say we have 20 VAVs. Staying on the same subnet, we have a controller here. This could be our advanced controller, which is doing our AHU, and then on this, we could start running our bus. So, we would have a controller, and each controller has two ports, one for the incoming Ethernet and one for the outgoing Ethernet. So, from an architecture standpoint, it's pretty simple.
With 21 devices here, we can start looking at what kind of subnet size we need. With 21 devices, the general rule of subnetting is 32 minus whatever. For 21, I’ll say 2 to the power of 5, which would give us 32 devices in an addressable range. So, we have 32 IP addresses; 32 – 29. That gives us a /29.
So, this is my CIDR, the size of my subnet. Right here, we're getting 32 IP devices. If you take away one for the default gateway and one for the broadcast address, you're going to still have plenty to address across here. A very simple bus and star hybrid architecture for a single building.
Now we're going to move into the concept of a ring architecture. So once again, we have a switch and our switch now is going to have three ports on it. In actuality, it has more than three ports, but three that we're utilizing. We would say we have an AHU once again, 20 VAV boxes. We would star off for AHU, and then for our VAV boxes, quite simply, we would start our ring architecture.
One port would be blocked, per Spanning Tree Protocol, and one port would be open. Now, we haven't talked about 802.1X, which is port level network security. We haven't talked about Spanning Tree and we're not going to talk a lot about spanning tree. We haven't talked about VLANs, and we haven't talked about routes.
But as we determined in this previous architecture, /27, which would give us 32 addresses. Two going to our default gateway and our broadcast address. This is building number one and we're going to have building number two over here. We're going to put another switch in here, and just for simplicity's sake, we're going to mirror this architecture. So, we're going to say we have an AHU and 20 VAVs.
Problem is, these are separate subnets, unless we set it up accordingly, which we're going to look at both ways. We're going to assume they're separate subnets here, and then we're going to talk about a VLAN to make them non-separate subnets.
So, we're going to have three ports once again, star pattern going to our AHU controller, ring pattern going to our terminal unit controllers.
Now the problem is, neither of these can communicate with one another, which is why we now have to introduce the concept of some layer three device, in our case it's going to be a router. This layer three device is going to have a route, and this is where that default gateway concept comes into play. So, what we would have to do is distribute our IP addresses, typically in coordination with IT.
We have 2-/27s on each side. For this one, we're going to do 192.168.1.0/27. Over here, we'll do 192.168.2.0/27. Then we'll have to have some link typically /30, right here, for the link between these two routers.
So, what will happen now is we'll start to distribute IP addresses. We'll have to give an IP address to the port on the router, this is going to be our default gateway, and we'll have to distribute addresses to each one of our devices. The same here as well.
So, what does this look like as a BAS professional. You either need to work with the IT group, or you need to start laying out on your network riser, what are the IP addresses? What are the subnets? Then, where do I need routes? Because remember, all of this stuff, where I have the red circle, that all is speaking on one subnet:
Where I have the second red circle, that's all speaking on a separate subnet, and they can't communicate with one another:
So, I'm going to assume you're using BACnet IP. We're going to have to have a BBMD and a BDT (BACnet Broadcast Management Device and Broadacst Distribution Table) We're going to have to have one on each subnet, so, we're going to have to account for that. What we're going to do is we're going to say, typically, our advanced controller for our AHU usually will have a BBMD built in, so it'll act as a BBMD, and we will have to populate the BDT with the address of each BBMD. We'll have to do that on each subnet.
Up to this point, pretty simplistic architecture, building one, building two, IP addresses, stuff like that.
Now, we're going to introduce maybe a larger building, and this larger building, maybe we have 500 VAV boxes, and 20 AHUs. Now, I've seen a lot of folks, the temptation with this is to be like, I’ve got 20 AUHs, 500 VAV boxes, /23, which would give you, in theory, 510 IP addresses. That's a horrible idea because now you have this massive network space to manage. So, you have to figure out how to manage this effectively.
What I tend to do in these situations is I tend to try to figure out is, what is the distribution of my IP assets, so my controllers? And then, where are the MDFs and IDFs? So, IDFs typically exist on each floor, usually near the elevator, and they run all the way to an MDF. This will typically sit right at the bottom of the building and be used as the distribution point back to the core network.
So, our IDF closets, which are going to sit typically on each floor, and our IDF will typically have our switch inside it. Inside our switch is where our ports are going to be. So, we’ll say we have 600 VAV boxes for the sake of even math, across six floors. So, maybe we’ll say we have 50 boxes on the first floor, and then every other floor has 125.
So, what we're going to want to do, is we're potentially, if it were up to me, have a VLAN per floor. I would have a subnet per floor. So in that case, with a subnet per floor, you're going to be looking typically at like a /25, which will give you 126 devices per floor. So, you'll have one spare address that you can plug your laptop into for configuration. So, what that would look like at each switch here, we would most likely use the ring architecture, and we would go out to our devices and we would come back.
This would be our /26. subnet. This would be on each floor, and then we would have to have one of these devices having a BBMD/BDT capability to interchange between these.
Now we would have a port, and all of these switch ports on each floor would most likely go back to a router. This router would have routes to each IP address on each floor.
So, what we would do here is, we may do something like 192.168.1.0, and then .2.0, .3.0, .4.0, and .5.0. These would be /25s. With this last one being 50, it would be /26, which would give us 64 addresses. With 64, that would actually give us 62 because remember, we always have to take one for the default gateway and one for the broadcast address.
So, all of these will go back to our router, which then would have routes in it. On each floor, we would have to have a BBMD on each subnet. So, you can see kind of how I'm building this out, it's not a terribly difficult concept.
Really, the hardest part of all of this is coordinating with the existing IT group on their subnets because they may not want to give out certain address pools to you. We recently had a customer reach out to us to help troubleshoot this very issue. So, what we decided was we got a BACnet router, and with this router, we had one incoming IP address. Then using PAT, which is port address translation, we were able to split it across four addresses. So, they were able to create their own /26 address pools that did not interfere with the customer's address pools, because the customer was giving them their own, /28 IP address range. Then, at the BACnet router, we were splitting this.
The nice thing about private addressing is that with port address translation, essentially, we take this address right here, and we give each one a port. So, we're able to divide it across that. Now, that's one way you can do things. It's a way to avoid having to ask for a ton of addresses from IT.
An additional thing that I see a lot of people do is they provide this infrastructure right here:
They provide the switch, they provide the router, and then they have a single port here, which connects to the customer’s IT network once the customer actually goes live with their networks. These would be industrial switches and industrial routers that you would put in place during the construction process.
So, this was a lot, but I hope it gave you a baseline understanding of what the architectures look like. You would diagram this out. Ideally, you would have a network riser diagram, but then beyond that, you would have a network list: Controller Name, Controller MAC Address, IP Address, and Port.
So, that's the physical port that it's connected to. In the case of a ring, you would just reference the port from the previous controller. So, you would say, Port one is from controller whatever. By building out this list, you then know what devices are what.
There are a lot more concepts in here that we didn't discuss. We didn't discuss something called DHCP and DNS – Dynamic Host Configuration Protocol and Domain Name Services. What we're seeing with a lot of newer controllers is that they are getting a domain name, basically a placeholder domain address. So, you would say like BAS.Penn.edu, and each controller would actually like be like, BAS1, BAS2, BAS3, etc. By giving a domain name, that decouples you from having to have a static IP address on that controller. So, if that controller changes, because it has a pool of addresses it can call from, then it doesn't matter because you're pointing at the domain name, not at the IP name.
This is going to become really important as a lot of people move from IPv4 to IPv6. IPv4 is a 32-bit address and IPv6 is a 128-bit address and it's represented in hex, not in numeric form. So, it's going to be a little bit confusing, which is why you see a lot of people moving to things like domain name services.
I hope you all found this helpful. If you have any questions, do not hesitate to follow up with us. I know this is a lot of information to digest and I don’t always go into greater detail because I could wind up going down a rabbit hole. I will say if you found this beneficial and you’d like to see a more structured, rigorous IT approach to things, check out our IT course.
Thanks a ton and take care.
