Turn PDFs into Clean Markdown for ChatGPT and RAG (Without Uploading Them)

If you've ever pasted a PDF into ChatGPT or Claude and got an answer about the wrong section, the model probably wasn't the problem. The text was.

A PDF doesn't store "a heading", "a table" or "a paragraph". It stores pieces of text placed at positions on a page. When you copy that text out, you get whatever order the PDF happens to keep it in, and the structure is gone. For an LLM, and even more for a RAG pipeline, structure is half the information.

This post covers what goes wrong, what clean Markdown for AI should look like, and a short Python chunker that splits it for retrieval while keeping page numbers.

Disclosure: I built QuickCatPDF, the free converter I use in the examples. The format and the chunking code work with any converter that produces similar Markdown.

What breaks when you copy text out of a PDF

Copy and paste from a typical report and you'll usually see some of these:

• Lines broken mid-sentence, because every visual line ends with a newline.
• Two columns mixed together, line by line.
• The page header and footer repeated on every page ("Annual Review 2026 · Page 7").
• Tables flattened into a run of numbers with no idea which value belongs to which column.
• Words split by hyphens at line ends ("docu- ment").
• Links gone. In a PDF, a link's address isn't part of the text at all. It's stored separately as an annotation on top of the page, so plain copy-paste only gives you the visible words.

None of this stops a model from reading the text, but it makes it worse at finding the right part and at quoting numbers from tables correctly.

What "clean Markdown for an LLM" looks like

Markdown is a good middle ground: models read it naturally, it's compact, and it keeps the structure that matters.

---
title: "Annual Review 2026"
source: "annual-review.pdf"
pages: 4
---

<!-- page 1 -->

# Annual Review 2026

## Introduction

Revenue grew in every quarter...

## Results by quarter

| Quarter | Revenue | Growth |
|---|---|---|
| Q1 | 1.2M | 4% |
| Q2 | 1.4M | 17% |

<!-- page 2 -->

A few details are worth copying, whatever tool you use:

1. Front matter naming the source file and page count. When you paste several documents into one chat, the model knows which is which.
2. Page markers as HTML comments (<!-- page 2 -->). Models read them, rendered Markdown hides them, and your RAG answers can cite a page.
3. Real headings (#, ##, ###) instead of bold lines.
4. Pipe tables, so every value stays in its row and column.
5. No page furniture: page numbers and repeated headers and footers removed, hyphenated words joined, wrapped lines merged into single paragraphs.

Three questions people keep asking

"How do I keep headings when converting PDF to Markdown?"
Most converters detect headings from the text's size and weight. That works when the PDF's headings are visibly larger or bold. If a document's headings are the same size and weight as the body text, there's no signal to detect, and they come out as plain paragraphs. In that case, add the ## markers by hand for the main sections. It takes a minute and makes a big difference for chunking.

"How do I keep tables?"
Pick a converter that outputs pipe tables, then check one table against the original. Tables without lines (columns separated only by white space, like bank statements) are the hardest case for any tool.

"Why are my hyperlinks lost?"
Because of the annotation point above: the address isn't in the text layer. A converter has to read the link annotations and match them to the words underneath. Plain copy-paste and many text extractors skip that step.

Chunking for RAG without losing page numbers

Headings are natural chunk boundaries, and the page comments let every chunk carry the page it started on. Here's a small, dependency-free chunker:

import re

def chunk_markdown(md: str, max_chars: int = 4000):
    """Split Markdown on ## headings, keeping the page number each chunk starts on."""
    md = re.sub(r"\A---\n.*?\n---\n", "", md, flags=re.S)  # drop front matter
    page = 1
    chunks, current, heading = [], [], None

    def flush():
        text = "\n".join(current).strip()
        if text:
            chunks.append({"heading": heading, "page": start_page, "text": text})

    start_page = page
    for line in md.splitlines():
        m = re.match(r"<!-- page (\d+) -->", line)
        if m:
            page = int(m.group(1))
            if not "".join(current).strip():
                start_page = page
            continue
        if line.startswith("## ") or sum(len(l) for l in current) > max_chars:
            flush()
            current, start_page = [], page
            if line.startswith("## "):
                heading = line[3:].strip()
        current.append(line)
    flush()
    return chunks

Each chunk comes back as {"heading": "Results by quarter", "page": 1, "text": "..."}, ready to embed. Store heading and page as metadata, and your answers can say "see page 1, Results by quarter" instead of pointing at the whole document.

If you prefer page-sized chunks (useful for contracts and forms), split on the <!-- page N --> comments instead.

Watch your token budget

A rough rule for English is about 4 characters per token, fewer for other scripts. That's close enough to decide whether a document fits in one prompt or needs chunking. Treat it as an estimate, not an exact count.

Doing it without uploading the PDF

Many PDFs you'd want to ask an AI about are exactly the ones you shouldn't hand to random websites: contracts, financial reports, internal docs. You'll decide later where the text goes. The conversion step doesn't need to send it anywhere.

The converter I built, QuickCatPDF's PDF to Markdown for LLMs, runs in the browser:

1. Drop in up to 20 PDFs.
2. Keep Front matter and Page markers as comments ticked.
3. Optionally limit the Pages.
4. Click Convert to Markdown, then copy or download. It shows the character count and a token estimate.

In my test, a 4-page report converted in about half a second into 4.6 KB of Markdown. Scanned PDFs are read with OCR in the browser too. You can check that nothing is uploaded: load the page, turn on airplane mode, and convert.

For plain Markdown for GitHub, Obsidian or Notion (with bold, italics, links and optional images), there's a separate PDF to Markdown tool.

Quick checklist

• Don't paste raw PDF text into an LLM when structure matters. Convert to Markdown first.
• Keep front matter and page markers, and drop the headers and footers.
• Check one heading and one table against the original.
• Chunk on ## headings and keep the page number as metadata.
• Estimate tokens at about 4 characters per token for English.

What's the worst PDF you've had to feed into an LLM? Tables, multi-column layouts, scans? I'm collecting hard cases.