Decentralized Location-aware Orchestration of Containerized Microservice Applications : Enabling Distributed Intelligence at the Edge

Abstract: Services that operate on public, private, or hybrid clouds, should always be available and reachable to their end-users or clients. However, a shift in the demand for current and future services has led to new requirements on network infrastructure, service orchestration, and Quality-of-Service (QoS). Services related to, for example, online-gaming, video-streaming, smart cities, smart homes, connected cars, or other Internet-of-Things (IoT) powered use cases are data-intensive and often have real-time and locality requirements. These have pushed for a new computing paradigm, Edge computing, based on moving some intelligence from the cloud to the edge of the network to minimize latency and data transfer. This situation has set new challenges for cloud providers, telecommunications operators, and content providers. This thesis addresses two issues in this problem area that call for distinct approaches and solutions. Both issues share the common objectives of improving energy-efficiency and mitigating network congestion by minimizing data transfer to boost service performance, particularly concerning latency, a prevalent QoS metric. The first issue is related to the demand for a highly scalable orchestrator that can manage a geographically distributed infrastructure to deploy services efficiently at clouds, edges, or a combination of these. We present an orchestrator using process containers as the virtualization technology for efficient infrastructure deployment in the cloud and at the edge. The work focuses on a Proof-of-Concept design and analysis of a scalable and resilient decentralized orchestrator for containerized applications, and a scalable monitoring solution for containerized processes. The proposed orchestrator deals with the complexity of managing a geographically dispersed and heterogeneous infrastructure to efficiently deploy and manage applications that operate across different geographical locations — thus facilitating the pursuit of bringing some of the intelligence from the cloud to the edge, in a way that is transparent to the applications. The results show this orchestrator’s ability to scale to 20 000 nodes and to deploy 30 000 applications in parallel. The resource search algorithm employed and the impact of location awareness on the orchestrator’s deployment capabilities were also analyzed and deemed favorable. The second issue is related to enabling fast real-time predictions and minimizing data transfer for data-intensive scenarios by deploying machine learning models at devices to decrease the need for the processing of data by upper tiers and to decrease prediction latency. Many IoT or edge devices are typically resource-scarce, such as FPGAs, ASICs, or low-level microcontrollers. Limited devices make running well-known machine learning algorithms that are either too complex or too resource-consuming unfeasible. Consequently, we explore developing innovative supervised machine learning algorithms to efficiently run in settings demanding low power and resource consumption, and realtime responses. The classifiers proposed are computationally inexpensive, suitable for parallel processing, and have a small memory footprint. Therefore, they are a viable choice for pervasive systems with one or a combination of these limitations, as they facilitate increasing battery life and achieving reduced predictive latency. An implementation of one of the developed classifiers deployed to an off-the-shelf FPGA resulted in a predictive throughput of 57.1 million classifications per second, or one classification every 17.485 ns.