Hey folks, Phil Zito here and welcome back. In this post, we are going to tackle the topic of the chicken and the egg, deploying IP devices to new builds. Now you may be like, “Okay, that makes no sense to me.” Well, if you've ever shown up on a job site to install IP controls, you've probably experienced what I'm about to describe to you.
You show up, you install your IP control devices, which every manufacturer out there is promoting right now. They're all telling you that you need to buy these products. So, you have these IP devices with you and install them. Then you realize, where do I connect them to? You start looking. (Cue the crickets). The IDF is not in, so the data closet is not in. The MDF, the main data closet, that's not in yet. No switches, no servers, no IT support. How in the world are you going to commission all of these IP devices on new builds? Now, those of you have done new build projects, you've probably run into this from time to time.
So, the first thing we need to do, in order to make sure we're all on the same page, is talk about the three primary IP device architectures. First off, in regard to IP devices, there are three primary architectures. Those are:
- Ring architecture
- Bus architecture
- Star architecture
Ring is probably the least familiar to you, but you can probably figure out what it is. It's a ring architecture. The devices are in a daisy chain, so it's Ethernet cable to Ethernet cable to Ethernet cable, and in between each Ethernet cable segment is a controller. At the end of the left side of the segment and the right side of the segment, is a switching device.
Then you have this fancy thing called Spanning Tree Protocol, or for some of you who are maybe using Cisco switches or switches that support this technology, rapid Spanning Tree Protocol. What Spanning Tree Protocol does is it eliminates this nasty little thing called a broadcast storm. So, basically IP devices communicate via broadcast, that's how they discover themselves. That's how they go back to the switch to say, “Here's my MAC address, and here's my IP address, pair us up and build out your ARP table,” your address resolution protocol table.
The problem is, using broadcasts, broadcasts go everywhere. Well, if you had a ring, broadcasts would go in circles and more broadcasts, and they would just never ever stop. You would have what is called a broadcast storm and the network would eventually crash.
So, how do we deal with that? Well, with Spanning Tree Protocol, you have what is called a blocking port. This blocking port is one of the physical ports, either on the right or left side, so, one of the switch ports that this ring is plugged into, and that blocking port blocks traffic. Now, as soon as that blocking port and switch detect that the other side of the trunk is down, the open port is communicating. That port, that switch, detects that the port is down for whatever reason, power, connectivity etc, and that port then is closed, and the blocked port opens and it allows for communication.
So, you get kind of the best of both worlds, right? You get the cost benefits of a bus architecture, or a daisy chain architecture, but you also get the redundancy of a ring. So, you get two ports, instead of one.
What I've seen a lot of folks do is connect to two different switches. So, they even have physical redundancy. We're going to talk about how that can definitely be a problem in the fact that there's no IP infrastructure and no IT infrastructure.
So, you have this rapid spanning tree, ring architecture. Now, you have the bus or daisy chain architecture. As you can imagine, it's simply starts off at a switch, and daisy chains along. Now, some supervisory devices enable you to run the daisy chain from the supervisory device, as do some advanced application controllers. They allow you to run the daisy chain, or the ring, from those controllers. It's kind of hit or miss, if that's your controller line. So, you need to research that.
The final architecture is the star architecture, or the one-to-one architecture. So, by the way, the bus architecture or daisy chain architecture is the lowest cost architecture. It also has a single point of failure at any of those controllers to bring down the chain.
What we have to realize, and this is still the case with a lot of manufacturers, is in a BAS controller, your MSTP bus, your BACnet MSTP serial bus, it is electrically bridges. So, it doesn't matter if the power goes down on your field controller, the RS 485, BACnet, MSTP, or FT10. Whatever signal is going to continue along on its merry way from controller to controller, so you lose power to the controller.
With how a lot of the IP controllers were built, they were built with individual network interface cards and software-based switching. So, as soon as that controller’s power goes down, that IP signal does not continue along. So, that's one of the downsides of IP controllers. Now, manufacturers are aware of this, and they're starting to bridge those network interface cards to allow those signals to continue in the case of power loss. But that is something to be aware of with both ring and bus architecture.
Now star architecture is one-to-one. It's one switch port to one device. This is what we most closely associate our supervisory device networks with. So, we have a supervisory device that has an IP drop, and we do a one-to-one from one switch port to that device. Now, the nice thing is, if that network were to go down, we only lose one device, right? So, we only have one device that goes out.
That being said, it's extremely expensive, because we are using switch ports. Some switches on the commercial IT side can cost 1000s of dollars. So, you're talking hundreds of dollars per port. That can be quite pricey if you're doing a star pattern to every single one. Plus, there's a lot of waste in having to run cables throughout the building to get to these end devices instead of doing a ring or daisy chain.
Okay, so now that we're familiar with architectures, let's talk about how we deploy these things. Standard deployment practices, we create a network riser, we figure out which of the three architectures we're going to use, and we figure out where we're going to make our runs to. So, are we going to run to this IDF closet, this MDF closet? How are we going to make our runs?
Then we need to decide on an IP addressing schema. Now, usually we would turn to the IT group for this IP address and schema but the reality is, IT switchgear is the last thing that is typically put in a building. So, their switches, the routers are usually not put in the building till the very end, and then IT has to configure them. So, getting an IP address schema is something that we're probably not going to have.
So, we need to figure out our schema. We need to understand how to subnet.
- Are we doing a slash 24?
- How many devices do we have?
- How big do we want this broadcast and collision domain to be?
- Do we want to implement VLANs, separate subnets, per building per floor?
What do we want to do? This used to never be an issue, right? You had maybe one sub net for an entire building because you had just supervisory devices and maybe a plant controller. Now, when you have terminal unit IP controllers, and everything has an IP address, we start to run into some concerns. I've seen people doing like slash 23, slash 22 networks, which that's 5,012 hosts or 1,024 hosts on those networks. Now, granted, it's not that many hosts because you have broadcast address, and all that default gateway, and all that stuff, but you're talking about huge collision domains and huge broadcast domains that could be a nightmare to manage.
So, my first kind of coaching point here is, don't go beyond slash 24. No slash 23 IP networks, those are bad. They're not impossible to troubleshoot, but they're very difficult to troubleshoot.
So we've laid out our architecture, we built our network riser, our electricians are starting to pull the Ethernet cable, they're starting to connect up our devices. We're starting to deploy our IP addresses. Let's go connect them to the IT switch…Oh, wait,
there's no it switches. So, what are you to do?
Well, this used to be quite a difficult conundrum, because the answer would have been to grab a 16 port Netgear switch. Let's just let the IP addresses do their thing, and they're not going to be interconnected, so we'll commission segments at a time, right? We'll do you know, Floor 1, Floor 2, Floor 3.
The problem with that was twofold. One was, if you had any logic that like plant dependent or air handler dependent, and you needed to internetwork between switches, it became quite a nightmare. You found yourself having to use managed switches to do this.
Enter the industrial switch. The industrial switch allows you to have level Layer 3 and Layer 2. So, Layer 3 is what we really need, it’s that routing capability to go between subnets. Because remember, we don't want to have multiple 512 or 1024 subnets. That's going to be a problem. So, if we have to have multiple slash 24 subnets, and we want the devices to communicate with one another, we have to have some form of routing.
So, Layer 3 industrial switch kind of fits the bill and they're not terribly expensive. Now granted, if you're using a star pattern, that can become a major issue, because you're going to eat up those switches. The industrials usually have 8-16 ports, tops. So, you're going to eat up your switch ports very quickly if you try to do star pattern. So, usually you're going to do daisy chain, maybe ring pattern.
The problem with those industrial switches and ring pattern is setting up rapid spanning tree or Spanning Tree Protocol on those switches, if you don't know what you're doing. That's usually something that's done by the IT folks, or it's built into your controller product.
Now the thing is, we have these devices laid out. We've put in either our Netgear switches, which I don't necessarily recommend. They're unmanaged, which means they're typically Layer 2, they have no command line interface, they have no way for you to manage them, no way for you to set up VLANs, no way for you to set up routes, no way for you to manage your network, hence the term managed switch or unmanaged switch. So, while IP traffic can go across Layer 2, it's actually going across in the form of Ethernet frames. It's not IP packets, it's Ethernet frames, and it's on that local physical LAN and subnet. It has to be a physical LAN unless you're using VLANs, in which case, you need Layer 3 switches, and it just becomes a nightmare.
So, here's the thing, you have your physical LAN, you have your switches set up, you have all of that communicating, but you have to find a way to connect. As I mentioned, the industrial switch allows you to do that, because it's a managed switch, and it's Layer 3. However, you have to understand how you go about actually setting up the routes. How do I do all this? So, you need to get a product, and I'm not going to make product recommendations, but you need to get a product that is easy to use. Ideally it has a graphical command line interface instead of a textual command line interface.
Okay, so we have our IP devices deployed, we have everything communicating to a point, probably not ideally how we want it to be on the production network, production meaning the final network, but for a deployment network, it's okay. Now the IT group comes in and this is where things get a little difficult.
You need to have a cut over strategy. So, you have to have a strategy, now, physical cutover, how are you going to rewire these switches? Ideally, these industrial switches, or Netgear switches, are in the same data closet as the IT switches. So, it's just a matter of having a coiled-up extension of the wire and just disconnecting and running. Now this will take your network down temporarily, so you need to be cognizant of that and you need to plan accordingly.
Additionally, what we also have to account for is, what if the IT group comes in and says that you used 192.168.1, but they want 192.168.10 for your slash 24 subnet. Now you need to re-address everything. Now, a lot of controls manufacturers support bulk readdressing, but that still is a problem. So, these are things you need to start to think about:
- How am I going to stage these things down?
- How am I going to re-address?
- Do I need to reroute things from a logical standpoint?
- Do I need to set up new routes?
These are all things that you need to consider when deploying IP devices. So, in a nutshell, deploying IP devices to building automation, it's not near as difficult as it used to be. That being said, there's a lot that you need to think of when you're going about deploying these things, and you need to be cognizant of them.
If you have any questions do not hesitate to reach out in the section below, as I know this can be confusing for some.
Thanks a ton and take care.
