I was on a live stream the morning I wrote this article and I was discussing my new book, with my pre-sale customers. A question popped up around data normalization, and data warehouses vs data lakes.
What a timely question. While I have focused so far on architecture and physical integrations, I do not want you to come away from this series thinking that data is unimportant.
It’s sometimes funny to me, that we focus so much on getting the systems integrated that we ignore the end result.
What is that end result you ask?
The result is data, that is presented in a format that we can take action on immediately.
Summary of Lesson #7
In this post, I am going to focus on how you can define the data model and system requirements.
By the end of this post you will understand the following:
- What is a data model and how do you create one?
- What are system requirements and how do you define them?
If you’ve been following the series up to this point, then you have identified the systems that will be used and the way in which the data will flow from system to system.
You also probably have an idea of what data you need. Now it’s time to dive much deeper and define the data model for each of the individual systems.
Now I will warn you, this will be quite an undertaking. Simply gathering information to create the data model is a challenge in itself. However, if you complete this step, you will avoid many of the data conflicts that cause systems integration to fail.
Once you have your data model defined, it will be time to define the system requirements. This should be rather easy considering that you have defined the systems in an earlier post.
Now you simply need to define the, hardware, software and network requirements.
Lesson #7: Detailing out the data model and system requirements
A brief segue on data
Data is everywhere. Some have said, “Data is the new oil”. Having the right “data” can provide you insight into how your building operates. With this information, you are able to make operational decisions that will impact the efficiency and effectiveness of your building automation system.
What is a data model and how do you create one?
A data model organizes data and details out how that data is formatted. When you are exchanging data between systems it is important to understand the format that the data is in. This is represented visually below:
As you can see I created a simple data model for a VAV box. This data model describes the primary key (which is how we will find the data) and two properties, zone temperature (ZN-T) and damper output (DPR-O).
You will notice that the primary key, controller name, is defined as the data type string and the two other properties are defined as the data type float.
In the programming world, a string, is a string of text and a float, is typically a number, with decimal precision.
Now, if I had, another system, that I was going to be integrating the VAV box data into and that system had a different data type then there would be a conflict. This is illustrated below:
As you can see, the VAV box data is not of the same data type as the third-party BAS data. This will cause a data conflict.
What is happening is the third-party BAS, has a primary key of device ID, which is of type int. An INT, stands for an integer. Unlike a float, an integer, does not have decimal precision. This means that the zone temperature and damper positions would lose their “precision” or accuracy.
But the bigger issue, is that the primary keys conflict both in type and in name. This will cause a mismatch. In order to resolve this the data needs to be normalized. You can see how data is normalized in the image below:
What I just did in the above image, is I created a higher level data set that bridges the two data sets utilizing what is called foreign keys. This allows the third-party BAS and VAV box data sets to access the data from the normalized data set.
When you purchase a device like a field server gateway, this is what is happening. Inside that gateway there is software that is performing this data normalization and then allowing the two data sets to connect.
When you look at an individual system you will want to go and create the data sets for each system. Now you may not know the data types that each system has. This is where you will have to turn to the system vendor to provide you that information.
Now I take a quick second the make sure I’m clear. If you’re performing a protocol integration using similar protocols, you most likely will not have to go this deep. The examples I am providing are geared more towards dissimilar protocols, for example integrating a LON system into a BACnet/IP system.
You can ask the vendor to provide you the data model for their specific system. Then you need to understand the points that you want to share between the systems. Once you have defined the points you will need to map up the data types.
Once you have done this you will need to determine if a normalization schema is needed. For some of you the word schema may be unfamiliar. When you hear the word schema, think format.
Schema is another way of saying the format of the data.
What are system requirements and how do you define them?
System requirements list out the physical and logical requirements of the systems that are part of the integration project. You will typically want to understand:
- CPU
- Memory
- Storage
- Bandwidth
- Network Throughput
In an earlier post we discussed how to identify your systems. Now that you know what your systems are its time to formally document the system requirements. This step is not very complex but it is time-consuming.
If you do not have an IT person on staff than is important that you retain the services of someone with an information technology background.
Here is a four step process for defining your system requirements:
Step 1: Define the network, storage, and compute requirements for each system
Using the list of systems that you created in (earlier post), you will need to go and work with your IT staff and vendors to determine the network (also known as bandwidth), storage, and compute (also known as processing) requirements of each individual system.
Step 2: Identify where the systems will be hosted
Based on the information from step one, you will want to work with your IT representative to determine where the systems will be hosted and what part of the network they will be connected to.
Step 3: Create an overall architecture diagram
Now that you have listed out where the systems will be hosted and the requirements you will need to create an overall architecture diagram. This architecture diagram will visually represent where the systems reside on the network and how they are connected to one another.
Step 4: List out any system, software, requirements
With all of this information defined, your final step is to work with your IT representative and your vendor to determine any software that is required to be installed on the systems in order for them to run.
Conclusion
There you have it. In this post, I have taught you how to model out your data and define your system requirements. We have one more post to go until this series is done.
In the next post, I will describe how to develop a sequence of operations for your integrated system.
Before I leave, I want to ask you what your processes for defining data models and system requirements. Post your answer in the comments section below.
