How I Structured 3,000 Prompts Into a Searchable Browser Tool

I had six PDFs containing 3,000 prompts, each with a title, a prompt body, camera notes, a mood and a "best for" line. To make them searchable in a browser, I needed them as clean structured data. Here's the pipeline from messy PDF text to a queryable in-memory dataset.

Step 1: Extract with structure intact

PDF text extraction (pdftotext) gives you a wall of text with headers, page numbers and footers interleaved. The first job is stripping the noise — repeated titles, Page N, copyright lines — before you try to parse the actual records.

def clean(text):
    lines = []
    for ln in text.split("\n"):
        s = ln.strip()
        if not s: continue
        if s.startswith("Page "): continue
        if s.startswith("© "): continue
        lines.append(ln)
    return "\n".join(lines)

Step 2: Parse records with a regex that respects the schema

Each prompt followed a consistent shape: a numbered title, then Prompt:, then Camera & Style:. A single regex with a lookahead to the next record captures each one cleanly:

card_re = re.compile(
    r"^#(\d+)\s+—\s+(.+?)\n"
    r"Prompt:\s*(.*?)\n"
    r"Camera & Style:\s*(.*?)"
    r"(?=^#\d+\s+—|\Z)",
    re.M | re.S)

The (?=^#\d+|\Z) lookahead is the key — it lets each record greedily consume its multi-line body up to the next record or end-of-file, without hardcoding line counts.

Step 3: Validate ruthlessly

With 3,000 records across six files, silent parse failures are the real risk. Assert the count per file — I expected exactly 500 each — and log which record numbers are missing if the count is off:

if got != 500:
    missing = [i for i in range(1,501) if i not in seen]
    print(f"  missing: {missing[:20]}")

Getting Vol 1: 500 OK six times is the whole ballgame. Without that check, you ship a library that's quietly missing 40 prompts and never know.

Step 4: A compact JSON shape

Short keys matter when the data ships inside the HTML file. {"v":1,"n":1,"c":"Sci-Fi","t":"...","p":"...","cam":"...","m":"...","b":"..."} keeps the payload small. Across 3,000 records, terse keys save real bytes.

Step 5: Query in-memory

Once embedded, "search" is just .filter() over the array across the fields you care about — title, body, category, mood. No index, no backend. At 3,000 records it's instant.

See the result

The finished tool is a single offline file. There's a free demo (36 records instead of 3,000, same structure) you can open and inspect if you want to see the shape in practice.

Try the free demo →