Can Self-Hosted CVAT Read Images Directly From Cloud Storage?

Self-hosted CVAT can read images directly from cloud storage. It does this through a built-in feature called Cloud Storages, which links CVAT to a bucket (a folder in cloud storage) and loads images from it whenever someone creates a task. This matters because most datasets are already sitting in cloud storage before labelling starts. Cameras, drones, and medical scanners save their files there by default, so by the time a labelling project begins, the bucket often holds tens of thousands of images. Downloading those images to a laptop and uploading them again through the browser is slow, it makes a second copy of the dataset, and you have to do it all over again every time a new batch of images arrives.

The connection removes that back-and-forth, but it does not remove the work around it. After the bucket is connected, four things still need looking after:

• Access keys that let CVAT open the bucket.
• A manifest file for large datasets.
• Disk space on the server, which keeps a copy of the images it shows.
• A step that copies finished label files back to the bucket.

On a few thousand images these are small jobs that come up once. On 50,000 images they come up every week. This article explains how CVAT connects to cloud storage, what the team manages as the dataset grows and how a pre-configured AMI on AWS Marketplace reduces that work.

How Does Self-Hosted CVAT Connect to Cloud Storage?

Self-hosted CVAT connects to cloud storage through the Cloud Storages page, where an administrator adds the bucket once by filling in the provider, the bucket name, the region, the folder inside the bucket, and the login details. After that, the bucket shows up as a choice every time someone creates a task, so labellers pick images from it instead of uploading them.

For large datasets, teams should check whether their CVAT version and storage setup require or benefit from a manifest file. The manifest's role in task preparation depends on the configuration, so check the relevant CVAT documentation before relying on it to speed up task creation.

All of this works, and it keeps working at any size. You set it up once and the CVAT project documents it well. What does change with size is the work behind the connection once the server is yours, and that work has four parts.

When Does CVAT Labelling at Scale Become a Problem?

Labelling at scale becomes a problem somewhere between 10,000 and 50,000 images, because that is the point where each of the four jobs changes from something you do once into something you do every week.

Below that range the work is light. Keys get changed once or twice a year, the manifest is built once and left alone, the cache fits easily on the disk, and an export is one small file that someone copies by hand. Above it, the jobs come up often enough to overlap, and that is when they start failing together: a large batch lands in the bucket, the manifest is already out of date, the cache fills the disk while the manifest is being rebuilt, and the access key expired last week. The table below shows how each job changes with dataset size.

Through all of this the connection itself keeps working. The strain falls on the four tasks behind it, each of which grows with the dataset.

What Does the Team Still Manage After Connecting?

The team still manages four things after connecting: the access keys, the manifest file, the server disk, and the export step. All four are easy to set up once, and all four turn into regular jobs as the dataset grows. The sections below take them one at a time.

1. CVAT Access Keys and Key Changes

A private bucket needs login details, and on a self-hosted server that usually means an access key that never expires, saved in the CVAT settings or on the server itself. Security rules say those keys must be changed from time to time, but every change breaks the storage connection, and labelling stops until someone updates the settings by hand. Faced with that, most teams do one of two things: accept the stoppage every few months, or quietly stop changing keys at all.

2. CVAT Manifest File for Large Datasets

The manifest is optional under 10,000 images and almost required at 50,000, because checking every file in the bucket without one takes too long to build a task. Building the manifest means running a script and waiting, and the result only describes the bucket as it was at that moment. When the next batch of images lands, the manifest does not know about them, so an engineer has to rebuild it, or labellers keep working from a list that is out of date without anyone noticing.

3. Server Resources and the CVAT Cache

CVAT keeps a copy of image frames on the server disk, called a cache, so the screen loads quickly. On a few thousand images that cache is small and nobody thinks about it. On 50,000 images, the cache may grow enough to put pressure on available disk space. When available disk space runs low, CVAT operations such as uploads and exports may fail. Teams should monitor storage usage and address capacity issues before they interrupt annotation work. Someone has to plan the disk size for this, and the same person usually also looks after updates, security certificate renewals, and the health of everything CVAT runs on: the container software, the database, the task queue, and the web proxy in front of it all.

4. CVAT Export to Cloud Storage

When labelling is finished, the default export is a file you download through the browser or that lands on the server disk. The training pipeline expects that file in the bucket, so an engineer has to set up a separate copy step and then remember to run it after every export. Miss it once and the training job picks up the old labels, and nobody notices until the model starts giving odd results.

None of these are faults in CVAT. They are the normal work of running the server yourself.

How Does a Pre-Configured CVAT AMI Reduce Setup Work?

A ready-made CVAT image reduces maintenance because it handles all four jobs at the server level, so the team uses the Cloud Storages feature and avoids managing the supporting setup themselves.

The CVAT AMI on AWS Marketplace from Yobitel comes with the same Cloud Storages feature as any CVAT install. You subscribe, launch the server in your own AWS account, open CVAT in a browser, and add the bucket exactly as you would on a server you built yourself. The difference is underneath, in how each of the four jobs is handled:

From subscribing to the first labelled image takes minutes, and the weekly upkeep of a self-built server does not come with it.

How Do You Get Started?

Start with Yobitel's CVAT AMI on AWS Marketplace by following these four steps.

1. Subscribe to the CVAT AMI on AWS Marketplace and launch it in your account.
2. Attach a permission that gives the server read and write access to your bucket.
3. Open CVAT in the browser, collect the admin password CVAT creates for you, and sign in.
4. Add the bucket under Cloud Storages and create your first task from a folder in it.

Full setup steps, server size suggestions, supported label types, and export formats are in the original post: CVAT on AWS: how Yobitel's pre-built AMI makes AI data annotation effortless at scale.

Self-hosted CVAT reads images directly from cloud storage, and the connection works the same whether the bucket holds 1,000 images or 1 million. The work around the connection is what grows with the bucket. Keys, manifest, cache, and export all need attention, and past 10,000 images that attention takes hours away from labelling.

The ready-made CVAT image on AWS Marketplace keeps the connection and removes the work around it. For a team whose dataset already lives in cloud storage and keeps growing, that is the reason to launch the image rather than build and look after the server. The questions below cover what readers most often ask before launching.

Frequently Asked Questions

1. Can self-hosted CVAT read images from cloud storage?

Yes, because an administrator connects the bucket once through the Cloud Storages page, and from then on its images are available as a choice whenever someone creates a task.

2. Does CVAT copy images to local disk?

Not as a full import, but it does read images from the bucket when someone creates a task and keeps a copy of frames on the server disk for speed, and on large datasets that copy can fill the disk.

3. Do I need a CVAT manifest file for a large dataset?

Not under 10,000 images, but at tens of thousands it becomes necessary, because checking every file in the bucket without one is too slow for task creation.

4. What is different about the ready-made CVAT image?

It runs the same CVAT with the same Cloud Storages feature, but access uses a server permission instead of saved keys, task creation and export run from ready scripts, and all the software is already running when the server starts.

Marketplace listing: Yobitel Image & Video Annotation Platform built on CVAT

Reference: CVAT documentation: Attach cloud storage