Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS)

Originally published on PkLavc.

Confirmed facts

On October 27, 2026, Amazon Elastic Container Service (ECS) announced support for GPU sharing using NVIDIA Multi-Process Service (MPS). This new capability allows multiple Amazon ECS tasks to run concurrently on a single NVIDIA GPU.

The primary objective of this feature is to increase GPU utilization and reduce infrastructure costs. It is specifically beneficial for workloads that do not fully consume a GPU's capacity, such as small-model inference, AV/ADAS simulations, embedding generation, and multi-model serving.

Engineers can implement GPU sharing without modifying their application code. The process involves defining the required GPU memory for each container and optionally setting a maximum share of GPU compute. Amazon ECS then handles the placement of tasks on GPUs with available capacity as services scale and deploy.

GPU sharing is supported on Amazon ECS Managed Instances and Amazon ECS on Amazon EC2. For Amazon ECS Managed Instances, the service includes monitoring of the MPS control daemon. If this daemon becomes unhealthy or stops, Amazon ECS automatically replaces the affected instance and migrates tasks to healthy capacity, ensuring workload continuity.

To begin using this feature, developers can utilize the AWS Management Console, AWS CLI, AWS SDKs, AWS CloudFormation, or other infrastructure-as-code tools. The setup involves creating a capacity provider with NVIDIA GPU instances, configuring the GPU sharing strategy and memory within the container definition of a task definition, and then creating a service on that capacity provider.

This capability is available across all AWS commercial and AWS GovCloud (US) Regions.

Technical analysis

The introduction of GPU sharing on Amazon ECS with NVIDIA MPS represents a significant evolution in cloud resource management for compute-intensive workloads, particularly in artificial intelligence and machine learning. Historically, a common challenge with GPU-accelerated workloads in containerized environments has been underutilization of expensive GPU hardware. Many inference tasks, for instance, do not require the full processing power or memory of a dedicated GPU, leading to inefficient resource allocation and increased operational costs. The explicit support for NVIDIA MPS directly addresses this by enabling fine-grained resource partitioning.

NVIDIA MPS is a runtime service designed to allow multiple CUDA applications to run concurrently on a single GPU, reducing the overhead associated with context switching and improving overall throughput. By integrating MPS, Amazon ECS can now manage and orchestrate these shared GPU environments. The mechanism described, where developers define GPU memory per container and optionally a compute share, indicates that ECS is abstracting the underlying MPS configuration. This abstraction simplifies the deployment process for engineers, allowing them to focus on application requirements rather than low-level GPU management. The ability to specify memory and compute shares at the container level within a task definition provides a declarative approach to resource allocation, aligning with typical cloud-native deployment patterns.

From an engineering perspective, the practical consequences are substantial. For organizations deploying numerous small-to-medium sized AI models for inference, embedding generation, or multi-model serving, this feature offers a direct path to significant cost savings. Instead of provisioning an entire GPU instance for each model or task, multiple tasks can now share a single instance, amortizing the hardware cost across more workloads. This is particularly relevant for microservices architectures where individual services might perform specific, less demanding AI functions.

Continue reading: Full technical article on PkLavc

Explore more technology articles: PkLavc Tech Blog

Telegram: https://t.me/PkLavcTech
Discord: https://discord.gg/zder5EZRK