Edge AI and DevOps: Orchestrating Intelligent Microservices at Scale

Abstract

Deploying artificial intelligence at the network edge promises low latency, reduced bandwidth, and data locality, but it breaks the assumptions that conventional DevOps and MLOps pipelines were built on: stable high-bandwidth links and homogeneous, always-connected hardware. This paper presents an orchestration approach for running intelligent microservices across a fleet of distributed edge sites at scale. We combine three ideas. First, a cloud control plane drives a lightweight edge orchestrator (e.g., KubeEdge or K3s) through GitOps desired-state reconciliation, so thousands of intermittently connected nodes converge to a declared configuration. Second, AI workloads are decomposed into containerized microservice pipelines (ingest → inference → post-process) and the inference model is optionally split across edge and cloud, with an optimal split point chosen to minimize end-to-end latency and edge energy. Third, model and code updates roll out progressively (canary) across the fleet with automated anomaly-gated rollback. We formalize the split-point and placement decisions, give the latency model, and report illustrative results, consistent with the literature, showing that collaborative split inference can beat both cloud-only and edge-only modes on latency while bounding edge energy. Public sources are cited throughout.