Dev.to Security πŸ” Cybersecurity πŸ‘ 0 πŸ“– 8 min read

Marketplace Object Storage: 6 Controls for Private Presigned Upload and Download URLs

Short answer: for a SaaS user avatar upload or marketplace training file, use private object storage, issue a short-lived presigned download URL only after authorization, and record retention as data rather than relying

Short answer: for a SaaS user avatar upload or marketplace training file, use private object storage, issue a short-lived presigned download URL only after authorization, and record retention as data rather than relying on somebody to remember a cleanup task.

The simplest object storage choice is the one that preserves this contract with the fewest moving parts. S3 compatibility can reduce integration friction, but it does not decide tenant authorization, object naming, browser CORS policy, retention evidence, or useful alerts. Those belong to the application boundary.

Delivery shape Pick it when Access-control cost Delivery cost
Private bucket plus presigned upload and download Browsers move ordinary avatars or training files directly and the app can authorize each intent The signing service must check tenant, actor, object, purpose, and expiry Low application bandwidth; one extra intent request
Application-proxied bytes Every byte must pass through application inspection or a synchronous policy decision Central policy is easy to see More application bandwidth, backpressure, and timeout handling
Public objects behind a cache The content is deliberately public and revocation delay is acceptable Weak fit for private artifacts Very simple reads and effective caching

That table is the field guide. For private marketplace material, start with the first row. Move to the second only when inspection or policy truly requires the application to carry the bytes. The third row is for public assets, not private-by-convention data.

Stage the private transfer rollout

Separate permission from transport. The browser first asks the application for an upload intent. The application authenticates the actor, confirms marketplace and artifact permissions, allocates an opaque object key, fixes the expected media type and size, stores a pending record with a retention class, and only then returns a short-lived presigned upload URL. After upload, a finalize request verifies the expected object identity before the record becomes readable.

Reads follow the same rule. A request for a private download URL names an application artifact ID, never an arbitrary storage key. The application resolves that ID inside the authenticated marketplace, checks the current policy, and signs a short-lived read. A copied URL can work only for its limited lifetime; deleting or changing the application record prevents fresh URLs from being minted.

Cost the control path in requests, not dollars

The diagram in words is: browser -> authorization service -> presigner -> private object storage for control, then browser -> private object storage for bytes. Logs come from the control path. Storage access telemetry covers the data path. Join them with an intent ID, not with a raw presigned URL, because URLs can contain credentials. Then measure control-plane decisions separately from data-plane outcomes and cleanup work. A useful signing event includes intent_id, artifact_id, marketplace_id, actor_id, operation, decision, policy_reason, and expires_in_seconds. Track upload intents created, denied, finalized, expired, and abandoned; declared bytes versus verified bytes; download grants by decision and artifact kind; and retention objects due, deleted, held, or overdue. This division matters during diagnosis. β€œUploads look slow” is a hunch. β€œThe p95 time from intent creation to finalize rose for 18–25 MB training files while signing latency stayed flat” points at the transfer path without blaming the signer. Alert on symptoms that require action: a sustained rise in authorization denials by policy reason, pending artifacts older than the upload window, retention deletions past their deadline, or a mismatch between declared and verified bytes.

Measure each boundary.

Keep it boring.

Browser uploads also need an explicit CORS policy. CORS is an HTTP-header mechanism enforced by browsers, and some cross-origin requests trigger a preflight request. Allow the exact web origins and methods the upload flow uses; don't treat * as an authorization system. The signature controls a storage operation. Your application still controls who may obtain that signature.

Use distinct error codes such as UPLOAD_SIZE_REJECTED, MEDIA_TYPE_REJECTED, and ARTIFACT_ACCESS_DENIED. A generic failure counter teaches the on-call engineer almost nothing. Test the policy as a matrix: Tenant A must never receive a grant for Tenant B's artifact; an expired intent must fail; a changed content type or size must fail; replaying finalize must be idempotent; a held artifact must survive its ordinary deletion date; and an unheld training artifact must become due exactly when its stored rule says it should. Then run the same contract suite against any candidate S3-compatible service before deployment.

How do presigned download URLs compare with proxy delivery for private SaaS user avatar uploads?

Pick direct presigned transfer when the files are routine avatars, course PDFs, or generated training artifacts and asynchronous validation is acceptable. It keeps the application out of the byte stream while leaving business authorization in one place. This is usually the cleanest balance between access control and delivery simplicity.

Pick an application proxy when policy must inspect every stream before storage or disclosure. The catch is operational: the service now owns upload duration, client disconnects, memory limits, backpressure, and bandwidth. That can be the right trade, but it deserves capacity tests and separate latency objectives.

Pick public cached delivery only for intentionally public output. It is not suitable when a marketplace seller expects immediate revocation, tenant isolation, or an audit trail for each grant. A hard-to-guess URL is still a URL, not an access-control decision.

S3-compatible behavior is useful as a portability boundary, but compatibility should be tested against the operations the system actually uses: signed upload, signed read, metadata lookup, deletion, CORS, and any multipart flow. I'm not sure a provider's compatibility label alone resolves edge semantics for every signing library; a contract test against the selected service does.

Implement the signed boundary with one state machine

The code below keeps vendor details behind a signer. It also makes the retention decision visible before any bytes move. The example is TypeScript end to end, but the important part is the sequence and the invariants.

type ArtifactKind = "avatar" | "training-file";
type RetentionClass = "profile-lifetime" | "course-plus-30-days";

type Actor = {
  id: string;
  marketplaceId: string;
};

type UploadRequest = {
  kind: ArtifactKind;
  contentType: "image/jpeg" | "image/png" | "application/pdf";
  bytes: number;
  sha256: string;
};

type SignedOperation = {
  url: string;
  expiresAt: string;
  headers: Record<string, string>;
};

interface ObjectSigner {
  signPut(input: {
    key: string;
    contentType: string;
    bytes: number;
    expiresInSeconds: number;
  }): Promise<SignedOperation>;

  signGet(input: {
    key: string;
    expiresInSeconds: number;
  }): Promise<SignedOperation>;
}

interface ArtifactRepository {
  insertPending(input: {
    id: string;
    marketplaceId: string;
    ownerId: string;
    objectKey: string;
    kind: ArtifactKind;
    retentionClass: RetentionClass;
    deleteAfter: string | null;
    contentType: string;
    bytes: number;
    sha256: string;
  }): Promise<void>;
}

const limits: Record<ArtifactKind, number> = {
  avatar: 5 * 1024 * 1024,
  "training-file": 250 * 1024 * 1024,
};

function retentionFor(kind: ArtifactKind, courseEndsAt?: Date) {
  if (kind === "avatar") {
    return {
      retentionClass: "profile-lifetime" as const,
      deleteAfter: null,
    };
  }

  if (!courseEndsAt) throw new Error("courseEndsAt is required");
  const deleteAfter = new Date(courseEndsAt);
  deleteAfter.setUTCDate(deleteAfter.getUTCDate() + 30);
  return {
    retentionClass: "course-plus-30-days" as const,
    deleteAfter: deleteAfter.toISOString(),
  };
}

async function createUploadIntent(
  actor: Actor,
  request: UploadRequest,
  signer: ObjectSigner,
  artifacts: ArtifactRepository,
  courseEndsAt?: Date,
) {
  if (request.bytes <= 0 || request.bytes > limits[request.kind]) {
    throw new Error("UPLOAD_SIZE_REJECTED");
  }

  const artifactId = crypto.randomUUID();
  const objectKey = [
    "marketplaces",
    actor.marketplaceId,
    "artifacts",
    artifactId,
  ].join("/");
  const retention = retentionFor(request.kind, courseEndsAt);

  await artifacts.insertPending({
    id: artifactId,
    marketplaceId: actor.marketplaceId,
    ownerId: actor.id,
    objectKey,
    kind: request.kind,
    ...retention,
    contentType: request.contentType,
    bytes: request.bytes,
    sha256: request.sha256,
  });

  const upload = await signer.signPut({
    key: objectKey,
    contentType: request.contentType,
    bytes: request.bytes,
    expiresInSeconds: 300,
  });

  return { artifactId, upload };
}

Six controls are doing real work here: private storage, authorization before signing, opaque tenant-scoped keys, fixed type and size, a five-minute signature, and persisted retention data. The digest supports later verification; it is not permission. A production finalize handler should compare stored expectations with trusted object metadata, transition pending to ready once, and reject attempts to finalize an artifact owned by another marketplace.

There is a subtle failure chain in this otherwise small flow. At 10:00, suppose the database accepts an intent with a five-minute signing window and a deterministic deletion date. The browser starts sending bytes, loses its connection, and never calls finalize. One branch leaves only a pending row; another can leave an object that the application cannot serve because its record is still pending. At 10:06, a scheduled reconciler can select expired pending records, inspect only their corresponding scoped keys, classify each as missing, abandoned, or finalize-late, and apply the stored retention rule. Every transition should carry the same intent ID and increment exactly one outcome counter. This is more than cleanup hygiene: without those states, an operator cannot distinguish a user who never transferred bytes from a successful transfer whose finalize request was lost, so retry guidance and deletion evidence both become guesses. Don't turn cleanup into an unobserved nightly task.

State beats hope.

Govern retention beyond a single PUT

For larger training files, multipart upload can improve recovery by sending parts independently. AWS documents that multipart uploads must be completed or stopped so stored parts no longer incur storage charges, and recommends considering multipart at roughly 100 MB. That makes lifecycle tracking part of the application model: persist the multipart upload identity, its owner, creation time, expected key, and completion state; alert on uploads that remain open beyond the intended window.

When should bytes stay on the application path?

Presigned direct transfer is not suitable when every byte requires synchronous malware scanning, transformation before persistence, or network controls that prohibit clients from reaching object storage. Stick with an application proxy or a dedicated ingestion tier in those cases. Public delivery is wrong for private training material even when it looks simpler.

Choose the service only after the contract test passes. The deciding evidence is small and concrete: private-by-default objects, correct signed PUT and GET behavior, the required CORS response, deterministic deletion, observable multipart cleanup if used, and operational limits that match artifact sizes. S3 compatibility narrows integration work. It doesn't replace the policy model.

References

πŸ“° Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.