Smart Office Azure Iot Predica Smart Office with Azure IoT and Cognitive Services

In my previous articles, I have introduced two important Azure Services: Digital Twins and Cognitive Services. This time, I would like to combine these services and present a real use case. I had the chance to implement a project called Predica Smart Office in our company’s headquarters. I will describe it in this post as an example of smart buildings solution.

This post is third in the series of articles on Azure IoT services. You can find my previous entries in this series here:

• Fusing digital and physical world with Azure Digital Twins
• Custom Vision AI with Azure IoT solutions

When talking about the Internet of Things (IoT) there are many different areas and scenarios where devices can provide valuable insights. IoT doesn’t have to be limited to industry and factories. IoT devices combined together with Azure IoT and Cognitive Services are a great solution to create smart buildings. That’s why we decided to use all of them at Predica office.

Predica Smart Office – answer to the specific problem

What problem are we trying to solve using this Smart Office solution? We have some conference rooms at our headquarters that we use for meetings. Moreover, it’s also a quiet place where consultants can make a call with the customers undisturbed. These conference rooms have different size and capacity. What is more, often after some meetings, conditions are not good enough to start another one – for instance temperature is too high and the air is not fresh.

To sum up, the main problem was that the conference rooms were used inefficiently (for instance small conf rooms were occupied when bigger ones were free) or someone booked a room but the meeting did not take place.

For that reason, we decided to implement a solution using IoT devices and services in the Azure cloud: Digital Twins and Cognitive Services.

Smart Office: solution overview

Predica Smart Office solution consists of desktop application, IoT devices and services in the Microsoft Azure cloud. In the application we can provide our office mapping (by uploading a file with specific structure). The next two tabs display information about two selected conference rooms.

In the space details section, we are displaying information about the conditions in the room (in this case, the temperature) and also how many people are currently in this specific room. What is more, we connected an IoT device with a button for ordering drinks. It’s enough to click the button to send the order to a dedicated channel on Microsoft Teams, so no one has to leave the room.

How does it work and how we get this information? We used four IoT devices. Two of them have many different sensors (including temperature sensor). The third one has a camera connected so we can take snapshots, send them to the cloud and analyze how many people are in the room. The last one is an IoT button device for sending coffee and tea order requests. In the following part of the article, I’ll provide more technical details about the implementation.

Smart Office solution architecture

To get information about conditions in each conference room and to check the number of people inside, we used Azure Digital Twins and Cognitive Services. Let me explain the components of the below architecture.

Azure Digital Twins

Let’s start with the elements on the left. There are two IoT devices connected to the Azure Digital Twins, which has an IoT Hub service underneath. If you are not familiar with this service, I encourage you to read my article about IoT services in the Azure cloud.

These two IoT devices have some sensors, eg. temperature sensor. In the Azure Digital Twins service, we defined User Defined Function (UDF) which is responsible for analyzing insights from specific sensors and on this basis providing clear information. In our case, we created UDF to analyze temperature data from the specific room, and provide information if it’s suitable to start another meeting or if the temperature is too high. Azure Digital Twins provides API, so it is possible to connect to it from the web, desktop or mobile applications to retrieve information about every room.

Universal Windows 10 Application

We wanted to provide easy visualization of the rooms together with information about conditions inside. This is why we created a Windows 10 desktop application. Once the user is logged in, there is an option to display information about two conference rooms in our office and check how many people are there and whether the temperature is good enough.

Azure Active Directory

The access to the application and Digital Twins Service is secured with Azure Active Directory. Users have to login first to be able to display information about conference rooms.

Cognitive Services – Computer Vision API and Logic App

As mentioned before, one of the IoT devices has a camera connected (right side of the architecture diagram). Snapshots are taken each specific period of time. Then the photo is sent to the Azure IoT Hub and stored in the Azure Storage in the blob container. There is a Logic App created to detect if there is any new image. Then the image is sent to the Computer Vision API to get information about the number of people recognized on the picture. The result is stored in the Azure Table Storage. Privacy is an important aspect here. Images are deleted immediately after the result of the analysis is returned by Computer Vision API. No face identification is applied to any person from the images so everyone stays anonymous.

Azure Function App

To enable a fast way to get information about the analyzed image, we created Azure Function. This one returns data from the Azure Table Storage to the desktop application.

Service Bus and Azure Logic App

As I mentioned before we have special IoT button that enables ordering coffee or tea. How does it work? Once the user clicks the button, a message is sent to the Azure IoT Hub. Then it is enqueued in the Service Bus service. After that, the trigger in the Logic App gets this message and sends it to the dedicated channel in Microsoft Teams.

Summary

This is only an example of the possible scenarios of how services in the Azure cloud can be used together with Cognitive Services to provide smart buildings solutions. Of course, there are more!

Smart Hotels

How services from the previously mentioned solution can be used by hotel managers? These are only some examples of how to improve hotel room service:

• Occupancy – detect how many people are in the room
• Light detection – alert when the room is empty but the light is on
• Temperature measurement – turn on air conditioning

Smart Restaurants

In the restaurants there is also a space for IoT solutions! Examples?

• Connected fridges – reports and alerts about the temperature
• Water consumption – information about water usage
• Kitchen power usage – measuring the power usage of devices

As you can see, there is a huge space for IoT solutions. With services like Azure Digital Twins and Cognitive Services you can get a lot of interesting insights which help you solve some specific problems, for example, with the conference rooms usage efficiency.

If you are interested in smart buildings, I encourage you also to watch the episode of Technical Fellows Show, where I show a demo of the solution and discuss it with Predica CTO!