SAMPLE EXERCISE ON CLOUD ARCHITECTURE

Data is all important in current digitized world. Data collected from assets and machinery provide insight into the operations of an industrial plant, vehicles, and heavy machinery.

Location data helps in tracking, guidance and route optimization for vehicle and people. Wearables can monitor real time health parameters of people.

Using sensors and cameras enormous amounts of data is collected from industrial equipment, vehicles, heavy machinery. These sensors will connect to gateways on the edge or in the cloud to send their data.

Collecting real-time data and monitoring all the assets requires a scalable solution which can be deployed on a distributed scalable infrastructure and platform services provided by popular hyperscaler cloud providers (AWS/GCP/Azure) or created in the data centers of organizations with the help of infrastructure and DevOps management tools.

IoT involves combination of technologies for the data collection and monitoring. It involves sensors and an edge gateway to read and collect real time data about the environment, machines parameters or location.

End to End IoT architecture

The following are the high-level requirements of a IoT solution which will be used to define the architectural goals for the solution:

This data is sent to the cloud where the applications provide users the ability to visualize the data and administrators use edge management services to manage the operations.

Data ingestion requires high throughput data extraction, transforming and loading (ETL). The data is stored in high speed scalable databases for visualization. Further data can also be used for ML and AI algorithms to provide statistical insight, automated alerts in real-time, forecast and prediction of operational parameters.

The solution needs to be capable of high number of concurrent users who would monitor data on dashboards and a limited number of administrative users who will be controlling and managing the devices remotely.

Based on the requirements the choice of architecture, technology and platform will vary. Here I have given a generic reference architecture for developing a IoT solution. With the above points in mind applications are designed with different user persona as applicable to the domain requirements.In the development of software for the internet of things (IoT) many of the architecture styles and patterns are applied to create a hybrid architecture.

In the development of software for the internet of things (IoT) many of the architecture styles and patterns are applied to create a hybrid architecture.

We see following sub architectures in this hybrid architecture:

IoT Gateway uses a microkernel architecture to enable connections to different sensors through protocol plugins/ This gateway software is developed to be suitable for edge deployment and operate in limited memory and compute restrictions.

Event broker pattern is implemented with the MQTT broker which is deployed in cloud and delivers messages from gateways to subscribers, i.e., a data pipeline.

Data pipeline can be implemented over EMB like Kafka, for execution of rules to check sensor data values and create alerts if an error is detected. Data is otherwise transformed and enriched with inputs from an ERP or MES system and stored in a data lake which is implemented as large column data base or a NoSQL document store.

Application services are implemented as microservices partitioned into the sub-domains

User interface is implemented using adaptive UI technologies which support development of a micro frontend and asynchronous events.

The analytics and machine learning pipeline can be deployed as a separate training and execution pipeline which works to train the models over the data set collected and the execution engine runs on the operational data pipeline to provide analysis and predictive output. There are subcomponents of a ML/AI pipeline which are not shown here.