Building a Whisper.cpp Container Image
Whisper.cpp is a high-performance, lightweight inference implementation of OpenAI's Whisper automatic speech recognition (ASR) model. It's written in plain C/C++ with no external dependency packages, and it performs spee
Whisper.cpp is a high-performance, lightweight inference implementation of OpenAI's Whisper automatic speech recognition (ASR) model. It's written in plain C/C++ with no external dependency packages, and it performs speech recognition with support for both CPU and GPU-based systems. This guide walks through cloning the Whisper.cpp repository and building container images for CPU-only and GPU-powered systems, then pushing the result to a container registry. By the end, you'll have a working Whisper.cpp container image tagged and pushed to a registry, ready to deploy.
1. Prerequisites
Before you begin:
- Deploy a GPU-enabled server with NVIDIA GPU drivers, CUDA, and Docker preinstalled.
- Access the server using SSH.
- Start the Docker service.
$ sudo systemctl start docker
- Add the non-root user to the Docker group. For example,
linuxuser:
$ sudo usermod -aG docker linuxuser
- Switch to the user:
$ su - linuxuser
2. Set Up the Server
1. Create a new directory to store your Whisper C++ project files:
$ mkdir whisper-project
2. Switch to the directory:
$ cd whisper-project
3. Clone the Whisper.cpp project repository using Git:
$ git clone https://github.com/ggerganov/whisper.cpp
4. List all directory files and verify that a new whisper.cpp directory is available:
$ ls
Output:
whisper.cpp
5. Switch to the project directory:
$ cd whisper.cpp
6. List all hidden files and verify that a .devops directory is available:
$ ls -a
Output:
. .devops cmake examples ggml-backend.h ggml-cuda ggml-impl.h ggml-metal.m ggml-quants.c ggml-vulkan.h .gitmodules models .......
The .devops directory includes the following Dockerfile resources:
-
main.Dockerfile: Contains the build context for CPU systems. -
main-cuda.Dockerfile: Contains the build context for GPU systems.
Use these resources in the next sections to build a CPU or GPU system container image.
3. Build a Whisper.cpp Container Image for CPU Systems
Use main.Dockerfile, which contains all necessary dependencies for CPU-based systems.
1. Copy the main.Dockerfile from the .devops directory to the main whisper.cpp project directory:
$ cp .devops/main.Dockerfile .
2. Build a new container image using main.Dockerfile with all files in the project directory. Replace whisper-image with your desired image name:
$ docker build -f main.Dockerfile -t whisper-image .
Wait for the build process to complete. When successful, your output should look like the one below.
[+] Building 117.7s (12/12) FINISHED docker:default
=> [internal] load build definition from main.Dockerfile 0.0s
=> => transferring dockerfile: 446B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 3.1s
=> [internal] load .dockerignore
3. View all Docker images on the server and verify that the new whisper-image is available:
$ docker images
Output:
REPOSITORY TAG IMAGE ID CREATED SIZE
whisper-image latest d54bd7163db5 4 minutes ago 608MB
4. Build a Whisper.cpp Container Image for GPU Systems
The main-cuda.Dockerfile resource contains the build context for GPU systems compatible with the latest NVIDIA CUDA drivers.
1. Copy main-cuda.Dockerfile to the main project directory:
$ cp .devops/main-cuda.Dockerfile .
2. Build a new container image whisper-gpu-image using the main-cuda.Dockerfile with all files in the working project directory:
$ docker build -f main-cuda.Dockerfile -t whisper-gpu-image .
3. View all Docker images on the server and verify that the new Whisper.cpp GPU image is available:
$ docker images
Output:
REPOSITORY TAG IMAGE ID CREATED SIZE
whisper-gpu-image latest 47be5b1566fd 31 seconds ago 2.76GB
To run the Whisper.cpp GPU container image, verify that your target host includes the minimum or higher CUDA version referenced by the ARG CUDA_VERSION= directive within main-cuda.Dockerfile.
4. Verify the target CUDA version:
$ cat main-cuda.Dockerfile | grep CUDA_VERSION=
Output:
ARG CUDA_VERSION=12.3.1
5. Upload the Whisper.cpp Container Image to a Container Registry
1. Open your cloud provider's console and navigate to your container registry to view the registry access credentials. Copy the Registry URL, Username, and API Key.
2. Switch to your server terminal session and log in to your container registry. Replace your-registry.example.com, exampleregistry, exampleuser, registry-password with your actual registry details:
$ docker login https://your-registry.example.com/exampleregistry -u exampleuser -p registry-password
3. Tag the Whisper.cpp container image with your desired registry tag. For example, your-registry.example.com/exampleregistry/whisper-gpu-image:
$ docker tag whisper-gpu-image your-registry.example.com/exampleregistry/whisper-gpu-image
4. View all Docker images on the server and verify that the new tagged image is available:
$ docker images
Output:
REPOSITORY TAG IMAGE ID CREATED SIZE
whisper-gpu-image latest 47be5b1566fd 6 minutes ago 2.76GB
your-registry.example.com/exampleregistry/whisper-gpu-image latest 47be5b1566fd 6 minutes ago 2.76GB
5. Push the tagged image to your container registry:
$ docker push your-registry.example.com/exampleregistry/whisper-gpu-image
When successful, your output should be similar to the one below:
The push refers to repository [your-registry.example.com/exampleregistry/whisper-gpu-image]
a42c530a140b: Pushed
445c20a8fa7a: Pushed
8ceb9643fb36: Pushed
latest: digest: sha256:70645eed0095268b6c11b051083cb83c41c1991dc088b0d0485ccc50aa4cf4af size: 2838
6. Open your container registry's management panel and verify that the new repository is available.
Next Steps
- Push the image to a container registry and wire it into your transcription pipeline.
- Benchmark CPU vs. GPU inference latency for your target audio workloads.
- Automate rebuilds via CI so new whisper.cpp releases stay current.
- Experiment with different Whisper model sizes for accuracy/speed tradeoffs.
For the full guide with additional tips, visit the original article on Vultr Docs.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.