Running a 39-Product Asset Pipeline on a 2 GB Single-Core VPS

The morning I knew the pipeline was broken, I was staring at a Dreamstime upload screen telling me my CSV header was invalid. Row one, column one. Nothing else. I had 39 finished products, 856 MB of output, around 19,000 lines of pipeline code behind them, and no idea which of the three channel formats I had mangled. That rejection cost me a morning and taught me the rule this whole build runs on: the platform format is the contract, and everything else is decoration.

Here is what actually works when you have one core, 2 GB of RAM, no GPU, and no patience for a second failed batch.

The four jobs that actually stall you

Making the art is the fun part. The four things that stop a solo maker are metadata, double posting, doc to product conversion, and packaging. Each one has a narrow, dumb solution.

For metadata, all three channels reject a wrong CSV header instantly. Adobe wants 5 columns, Vecteezy wants 4, Dreamstime wants 15. A small script that writes exactly that header, derived from a folder of files, removes an entire class of rejection.

For double posting, the fastest way to get an account flagged is running your batch twice. A plain ledger file that records what you already submitted, and refuses to hand you the same item again, makes that impossible. It also gives you a pending view so you always know what is left.

For turning a doc into a product, the content already exists. It just needs to look like something you would pay for. One command that turns Markdown into a clean PDF with headings, tables, lists, and links gets you there.

For packaging, a zip with a real cover sells. A bare file does not. So the pack step zips the files and renders a 1280x720 cover and a 600x600 thumbnail in the same pass.

One process, more RAM than you think

The RAM problem on a 2 GB box is not average usage. It is the peak. A single image job can look harmless and then spike when you least expect it.

Four habits made the whole pipeline fit on 1 vCPU:

• Run one process at a time. Never fan out image jobs in parallel on a single core. Parallelism on one core is just slower work with a higher memory ceiling.
• Call gc.collect() after every asset. It is one line and it keeps your footprint flat across a 39 product run.
• Measure peak RSS, not average. resource.getrusage(RUSAGE_SELF).ru_maxrss gives you the real number, and the real number is what kills a small box.
• Process one file per subprocess for heavy render steps. The OS gets the memory back when the process exits, which is more reliable than trusting your own cleanup code.
• Render at the resolution you need. A preview inspected at 60 dpi costs a fraction of the same preview at 300.

None of this is clever. It is the difference between a box that finishes the batch and one that gets killed mid-run with half a product folder on disk.

Two packages, no framework, no cloud account

The pipeline needs exactly two third party libraries. Pillow for the covers and thumbnails. Reportlab for the PDF output. The metadata builder and the upload tracker use the standard library alone, so they run on a bare Python with no install at all.

That matters more than it sounds. Every dependency you add is another thing that can fail on a fresh server at 2 AM, and another reason the pipeline needs a babysitter. Python 3.9, a virtual environment, pip install pillow reportlab, and you are done.

The folder layout is the other half of the discipline. Without one, a catalogue turns into a junk drawer within a month:

project/
  assets/      raw output, one file per asset
  output/      ready to submit files plus metadata.csv
  packs/       finished product zips and covers
  research/    notes and sources
  scripts/     the tools

Keep research/ forever. The scrape logs and notes are the raw material for future products. Deleting them to clean up is a mistake this project made once and will not repeat.

The rules behind the scripts

Every header, column count, and required field in the kit traces back to something concrete. Three channel formats. A fixed set of required metadata per platform. A cover size of 1280x720 and a thumbnail of 600x600. The kit enforces those because a wrong one costs you a rejection and a re-upload, and a re-upload batch is exactly how accounts get flagged.

That is also why the tracker sits in front of the upload step. The pipeline does not trust memory, yours or mine. It trusts a ledger file.

The 39 products, the 856 MB of output, and the 300 plus pages of research behind these rules were all validated on that same 1 vCPU, 2 GB, no GPU box. The numbers are not a benchmark. They are what this rig actually produced.

Where this came from

These four scripts are the packaging layer, not a framework. If you want the exact tools that turned one prompt engine into 39 finished products, they are packaged as the compliance metadata kit: the metadata builder for Adobe, Vecteezy, and Dreamstime, the upload tracker, the Markdown to PDF converter, and the product packer. Standalone Python, two dependencies, the same rules enforced.

Get compliance metadata kit

Building digital products? Free guides & kits for digital product sellers.