How to Build Auditable User Document Storage with Python — Access Control

A receipt system in healthtech has two jobs that pull in different directions: make a document easy for the application to deliver, and make the original file defensible during an audit. The design should keep those jobs separate. Store the original in private object storage, issue short-lived access only after an authorization check, and record an immutable metadata trail around every read and write.

Short answer: test the same storage contract against AWS S3, Wasabi, Backblaze B2, or another object store before committing to an adapter. The deciding question is not which bucket is easiest to create. It is whether your access-control boundary, original-file retention, upload recovery, and audit evidence still work when a request fails halfway through.

The example below uses Python and a generic S3-compatible interface. It is deliberately small enough to run in a notebook, then strict enough to become a service boundary. Before adopting any backend, measure authorization latency, failed-upload recovery, retrieval behavior, and the evidence produced for an audit.

What should the receipt workflow guarantee?

A receipt arrives with an owner, a record identifier, and an original byte stream. The system should preserve the bytes, associate them with a controlled record, and make later changes visible rather than silently replacing history. A useful record has at least:

• an opaque object key, never a patient identifier in a URL;
• the original filename as metadata, not as the storage identity;
• content type, byte length, and a cryptographic digest;
• uploader, tenant, record identifier, and creation time;
• a retention decision and an audit event for each access attempt.

The original file is the evidence. A resized preview, OCR text, or normalized PDF can be derived data, but it should not overwrite the original object. This distinction matters when an OCR job misreads a decimal or a later parser changes its output. Reprocessing should create a new derived artifact and leave the source addressable.

I initially treated a signed download URL as the whole delivery design. It was convenient, but the URL became the thing people reasoned about instead of the authorization decision. The correction is simple: authorize first, mint a short-lived URL second, and log both the decision and the delivery request.

Why is access control harder than a private bucket?

A private bucket is a useful storage setting, not an application authorization model. The application still needs to answer: may this user read this receipt, for this tenant, in this session, at this time? That decision belongs in the application or an access-control service that knows the business relationship.

The delivery path can then be intentionally boring. A service checks the request, finds the object key in its database, records an allow or deny event, and returns a short-lived delegated download. The browser or mobile client receives bytes without receiving broad storage credentials. For uploads, the same boundary can create a constrained upload request after validating the intended record and expected content type.

Three states are enough to make the first version legible: pending, available, and rejected.

There are two important failure cases. If authorization succeeds but URL creation fails, the audit event should say that delivery was not issued. If URL creation succeeds but the client never downloads, that is a delivery outcome, not proof that a human viewed the receipt. Use separate event names; otherwise an audit report can overstate what happened.

A small policy function makes this boundary testable:

from dataclasses import dataclass
from datetime import datetime, timezone


@dataclass(frozen=True)
class Receipt:
    tenant_id: str
    record_id: str
    object_key: str
    digest: str
    byte_length: int
    created_at: datetime


def can_read_receipt(
    *,
    receipt: Receipt,
    requester_tenant: str,
    requester_record: str,
    now: datetime,
) -> bool:
    if receipt.tenant_id != requester_tenant:
        return False
    if receipt.record_id != requester_record:
        return False
    if receipt.created_at > now:
        return False
    return True


now = datetime.now(timezone.utc)

This code does not authenticate anybody, and it does not replace a policy engine. It does show the decision inputs explicitly. That is valuable in tests: a tenant mismatch, record mismatch, and future-dated record should each have a named case, not a vague assertion about a response status.

The upload path: preserve bytes, then publish metadata

Uploads should have a clear state transition. A database row can begin as pending, the object can be uploaded and verified, and the row can become available only after the digest and byte count match the intended file. If the process dies between those steps, a repair job can find pending rows without guessing what an orphaned object means.

Large files add another concern. Multipart upload lets an object be sent in parts and completed later; the storage service's multipart documentation describes the upload, part, and completion lifecycle. That lifecycle is not the same as an application transaction. Your database may commit while the upload is incomplete, or an upload may complete while the metadata transaction rolls back. Design a reconciler for both directions.

Here is a provider-neutral shape for the verification step. The storage client is injected so the policy and checksum logic can be tested without a network connection:

from hashlib import sha256
from typing import BinaryIO, Protocol


class ObjectReader(Protocol):
    def open(self, key: str) -> BinaryIO: ...


def digest_stream(stream: BinaryIO, chunk_size: int = 1024 * 1024) -> tuple[str, int]:
    digest = sha256()
    length = 0
    while chunk := stream.read(chunk_size):
        digest.update(chunk)
        length += len(chunk)
    return digest.hexdigest(), length


def verify_original(
    *,
    reader: ObjectReader,
    object_key: str,
    expected_digest: str,
    expected_length: int,
) -> None:
    with reader.open(object_key) as stream:
        actual_digest, actual_length = digest_stream(stream)
    if actual_digest != expected_digest:
        raise ValueError("original digest does not match")
    if actual_length != expected_length:
        raise ValueError("original length does not match")

A one-megabyte read size is an implementation default, not a universal benchmark. Test it with the files your system actually receives. The key invariant is stronger than performance: the record must not become available merely because an upload request returned successfully.

How should user document storage balance access control and delivery?

AWS S3, Wasabi, and Backblaze B2 can all appear in a shortlist for object storage, but the shortlist is not an architecture. Treat each as an adapter behind the same contract. Keep the application-facing vocabulary to operations such as put_original, open_original, start_multipart, complete_multipart, and delete_after_retention; keep provider-specific request details inside the adapter.

The meaningful comparison is observable behavior. Can the adapter enforce private access? Can it produce the delegated delivery mechanism your client needs? Does an interrupted multipart operation remain discoverable and cleanable? Can the team inspect access events and reconcile object inventory with database rows? The answers should be recorded from current documentation and a small test account, not inferred from a familiar API name.

This table is a test plan, not a ranking. A backend name does not answer the authorization question.

I would start with three checks: can the wrong tenant never read the object, can an interrupted upload be found, and can an auditor join each access event to a receipt row?

The storage layer should stay boring.

Auditability is a workflow property, not a bucket label.

Delivery simplicity may look attractive when a client can fetch an object directly. In a healthtech workflow, that simplicity can move authorization into a place that has less context. I would accept a slightly more involved delivery path when it keeps tenant and record checks in one service, because the operational cost is visible and testable. I would not accept hidden credentials in a client just to remove one server hop.

This is also where compatibility claims need care. An S3-shaped API can reduce adapter work, while still leaving differences in multipart behavior, metadata handling, lifecycle controls, error details, and access-log integration. Compatibility reduces syntax risk; it does not prove equivalent compliance evidence or failure recovery.

A compact evaluation harness before production

The evaluation should use representative receipts, including a small image, a multi-page PDF, a truncated upload, a duplicate submission, and a file whose content type is misleading. Run the same cases against every candidate adapter. Record results as structured data so a notebook experiment can become a CI check later.

from dataclasses import dataclass
from typing import Protocol


class StorageAdapter(Protocol):
    def put_original(self, key: str, data: bytes, content_type: str) -> None: ...
    def read_original(self, key: str) -> bytes: ...
    def remove_original(self, key: str) -> None: ...


@dataclass
class CaseResult:
    name: str
    passed: bool
    detail: str


def run_case(adapter: StorageAdapter) -> list[CaseResult]:
    results = []
    payload = b"receipt-test"
    key = "tenant-a/record-17/original"
    adapter.put_original(key, payload, "application/octet-stream")
    results.append(CaseResult("round trip", adapter.read_original(key) == payload, "bytes"))
    adapter.remove_original(key)
    return results

The harness above is intentionally not a benchmark. Add measurements where they inform the decision: authorization-to-URL latency, upload completion time by file size, recovery time for abandoned multipart uploads, and the percentage of audit events that can be joined to a receipt row. Keep the test data synthetic. A healthtech storage evaluation does not need real patient documents to expose an adapter boundary.

A useful release gate is boring: every candidate passes the same contract tests, denied cross-tenant reads stay denied, incomplete uploads do not become available, and reconciliation reports are explainable. If one backend is easier to deliver from but produces weaker evidence, that is an access-control trade-off, not a small implementation detail.

The decision rule

Use the contract results to identify which backend satisfies the evidence requirements with the fewest special cases your team can operate. Let delivery simplicity influence the decision only after private access, authorization placement, original-file integrity, retention, and reconciliation are proven. Keep the adapter replaceable, but do not pretend replacement is free: migration needs a digest-preserving copy, a dual-read or cutover plan, and an audit record for each move.

The right result is not a universal winner among storage brands. It is a receipt workflow where a reviewer can answer what file was stored, who could request it, which authorization decision allowed delivery, and whether the bytes still match the original. Measure those answers before copying the architecture into production.

Further reading

• AWS S3 Multipart Upload overview: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
• AWS S3 User Guide: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html
• Wasabi documentation: https://docs.wasabi.com/
• Backblaze B2 documentation: https://www.backblaze.com/docs/cloud-storage
• NIST Digital Identity Guidelines: https://pages.nist.gov/800-63-3/