Private User Document Storage for PDF, DOCX, Invoices, and Tenant Metadata
Short answer: store uploaded bytes in private object storage, but make a tenant-scoped database record the only way an application can discover or deliver them. Prefixes organize keys; they do not authorize a download.
Short answer: store uploaded bytes in private object storage, but make a tenant-scoped database record the only way an application can discover or deliver them. Prefixes organize keys; they do not authorize a download.
For a product handling user uploads, the decisive question is not whether the object is a PDF, DOCX, or invoice. It is where the system makes its access decision. Put that decision in a query constrained by the authenticated tenant, keep generated identifiers separate from user-supplied names, and treat the object store as a byte service. That gives a Python service a small control plane to test before a document reaches a parser, an embedding job, or a model prompt.
Trust the query.
Start with an upload state machine, not a folder convention
Use one logical document row for each upload. It should carry the tenant ID, uploader, opaque document ID, original filename, claimed media type, byte count, checksum, object key, and lifecycle state. The original name is display data. It is never a canonical key segment or an authorization input.
The object key can still be legible enough for operations: tenants/{tenant_id}/documents/{document_id}/v1. A prefix helps inventory, targeted retention work, and incident investigation. It does not stop a caller who has somehow obtained a key. Private storage plus a scoped metadata lookup does.
For an AI workflow, preserve the distinction between source bytes and derivatives. Parsed text, chunks, embeddings, and generated summaries each need a reference to a specific checksum or document version. Otherwise a retry can quietly parse and tokenize identical bytes again, which makes cost attribution and evaluation results fuzzy.
Keep the states few: pending, ready, quarantined, and deleted are usually enough. Readers see only ready; a deletion request makes a record ineligible before cleanup work removes the bytes. Small rules, large payoff.
| Concern | Authoritative location | Why it matters |
|---|---|---|
| Object bytes | Private object storage | It keeps large payloads out of ordinary database reads and backups. |
| Tenant ownership and state | Relational metadata | A scoped query can decide access before any object key is used. |
| Original filename | Metadata | It is presentation data, not a storage address. |
How should a multi-tenant SaaS protect private PDF, DOCX, and invoice uploads?
The service should derive tenant_id from server-side authentication and generate document_id itself. It must not accept either value from the browser as a trusted storage location. On upload, write the object, then create the metadata row; if the metadata transaction cannot commit, remove the object. On download, query by both document ID and tenant ID before requesting bytes. Do not fetch by ID and check ownership later.
Here is a compact Python version. The in-memory object store keeps the important ordering and authorization behavior visible without tying the article to a provider.
from __future__ import annotations
import hashlib
import sqlite3
import uuid
from dataclasses import dataclass
@dataclass(frozen=True)
class AuthContext:
tenant_id: str
user_id: str
class PrivateObjectStore:
def __init__(self) -> None:
self._objects: dict[str, bytes] = {}
def put(self, key: str, body: bytes) -> None:
self._objects[key] = body
def delete(self, key: str) -> None:
self._objects.pop(key, None)
def create_database() -> sqlite3.Connection:
db = sqlite3.connect(":memory:")
db.execute(
"""
CREATE TABLE document_metadata (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
uploaded_by TEXT NOT NULL,
original_name TEXT NOT NULL,
byte_count INTEGER NOT NULL,
sha256 TEXT NOT NULL,
object_key TEXT NOT NULL UNIQUE,
state TEXT NOT NULL CHECK (state IN ('pending', 'ready', 'deleted'))
)
"""
)
return db
def save_upload(
auth: AuthContext,
original_name: str,
body: bytes,
objects: PrivateObjectStore,
db: sqlite3.Connection,
) -> str:
if not body:
raise ValueError("an upload must contain bytes")
document_id = str(uuid.uuid4())
key = f"tenants/{auth.tenant_id}/documents/{document_id}/v1"
digest = hashlib.sha256(body).hexdigest()
objects.put(key, body)
try:
with db:
db.execute(
"""
INSERT INTO document_metadata
(id, tenant_id, uploaded_by, original_name, byte_count,
sha256, object_key, state)
VALUES (?, ?, ?, ?, ?, ?, ?, 'ready')
""",
(document_id, auth.tenant_id, auth.user_id, original_name,
len(body), digest, key),
)
except Exception:
objects.delete(key)
raise
return document_id
def locate_for_download(
auth: AuthContext, document_id: str, db: sqlite3.Connection
) -> tuple[str, str] | None:
row = db.execute(
"""
SELECT original_name, object_key
FROM document_metadata
WHERE id = ? AND tenant_id = ? AND state = 'ready'
""",
(document_id, auth.tenant_id),
).fetchone()
return None if row is None else (row[0], row[1])
This is deliberately plain. The paired predicates are the important line: a cross-tenant ID has no matching metadata record, so the delivery path never receives an object key. A security test should create two tenants, upload under one, and assert that every public read operation returns no document for the other. Run it against list, download, preview, extraction, and deletion paths, because an overlooked secondary endpoint is where isolation tends to erode.
The awkward gap between object storage and SQL
An object store and a relational database do not share one transaction. The code above handles the common failure window by compensating: delete a freshly written object when metadata insertion fails. For large or direct-to-storage uploads, make the gap explicit with a pending row and a finalize operation. A finalizer verifies the expected checksum and object key, then transitions the row to ready in the database transaction.
The failure sequence deserves more attention than the happy path. A client can finish sending bytes and disappear before finalization; a worker can retry after the browser has already shown success; a deletion can arrive while extraction is waiting; and a deployment can point a worker at the wrong environment. None of those cases should make an unscoped object discoverable. Record the intended tenant and document ID before accepting an asynchronous upload, require the finalizer to compare them with the expected key and checksum, and let only the metadata state expose the upload to a list or download handler. When a retry repeats, it should observe the same logical document rather than manufacture another one. This is also a useful place for an eval harness: exercise each state transition with a second tenant and confirm the transition never widens the query.
The catch is that this pattern is not suitable when a regulatory or contractual control requires a separately administered storage boundary for each tenant. In that case, use isolated storage accounts or buckets and accept the extra provisioning and policy work. Database binary columns can also be the better fit for very small, transaction-bound artifacts, although backups, replicas, and normal query traffic then carry the bytes too. There is no universal size cutoff; measure restore time, read volume, and operational ownership in the actual deployment.
Make reconciliation a normal job, not an emergency script. It can identify aged pending rows, metadata whose objects are absent, and objects with no metadata owner. Cleanup and deletion must be idempotent, because queues retry. The same mindset applies to document ingestion: key parser and embedding work by the source checksum, then test that repeating an unchanged upload does not create a second derivative set.
Treat delivery, caching, and telemetry as part of storage design
Browser upload progress is useful feedback, but it is not proof that a completed object belongs to a user. The server still performs the authenticated metadata transition. Downloads need an equally deliberate response policy: private documents should start with conservative Cache-Control behavior, then receive integration tests for the browser and intermediary caches used by the application. Cache directives have different effects depending on the cache and response context, so validate the headers on real private download responses rather than copying a snippet from a public asset setup.
Don't log document contents, raw authorization headers, or signed delivery material. Log a request ID, a tenant-safe internal identifier, document ID, lifecycle transition, byte count, and duration. Those fields can explain an upload path without turning observability into another document repository.
Before release, exercise a duplicate filename under two tenants, an attempted cross-tenant read, a failed metadata write after object creation, a repeated deletion, an old pending upload, and a restore into an isolated environment. Then run the extraction and retrieval evaluation twice against the same checksum. The second pass should be boring. Good.
References
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.