How a 24-Hour Apache Pulsar Outage Turned Into a Local-First Web Toolkit
A few months ago, I spent 24 hours straight tracking down a production Apache Pulsar outage: producers kept writing to a topic, yet every consumer on that topic froze. The deceptive bug: failed to read entries
A few months ago, I spent 24 hours straight tracking down a production Apache Pulsar outage: producers kept writing to a topic, yet every consumer on that topic froze.
The deceptive bug: failed to read entries wasn't a storage failure
The broker logs pointed straight at storage (EntryCacheImpl - failed to read entries), which normally makes you suspect BookKeeper data corruption.
Reading the BookKeeper ledger directly proved the entries were intact. Comparing Pulsar 2.8.4 against 2.10.6 finally exposed the real culprit: a rare same-thread recursive reentrancy on the replay/redelivery path:
ManagedCursor replay
-> ManagedLedger asyncReadEntry
-> EntryCacheImpl cache hit (synchronous callback)
-> PersistentDispatcherMultipleConsumers readEntriesComplete
-> sendMessagesToConsumers
-> readMoreEntries -> asyncReplayEntries -> ... -> StackOverflowError
When multiple conditions lined up for enough consecutive rounds, readMoreEntries() kept recursing on the same BookKeeperClientWorker-OrderedExecutor thread until it blew the JVM stack. Pulsar 2.10.6 later added sendInProgress reentrancy guards and async executor scheduling specifically to prevent this.
The real pain after the outage: fighting Word while exhausted
Stabilizing the cluster was only half the battle. By the time the postmortem was ready, I was completely drainedβand still had to deliver clean, human-readable Word (.docx) reports to multiple non-engineering departments.
What I had on my desk:
- Piles of Markdown notes and AI-assisted root-cause drafts full of dense jargon.
- Production log snippets with internal IPs, hostnames, and service names.
- Timeline tables, formulas, and Mermaid diagrams.
Anyone who has tried copy-pasting Markdown or AI output into Word knows the headache: tables collapse or misalign, equations turn into flat text or blurry screenshots, Mermaid blocks stay as raw code, and you end up spending another hour fighting indentation, line spacing, and styles when all you want is to go to sleep.
And because the drafts contained real production logs, uploading them to random online converters was out of the question.
What I built out of that night
That single night became the core workflow behind Pathadd β a restrained, browser-only toolkit with zero sign-up, zero ads, and zero uploads:
- Log Redaction & Restoration: Mask IPs, tokens, and connection strings locally with reversible tokens before asking AI for help, then restore real values in one click.
- Redacted AI Context Packer: Pack multiple local source files into structured Markdown with token estimates, default secret masking, and custom KβV replacements (e.g., internal domains or company names).
-
Markdown & AI to Editable Word (
.docx): Convert Markdown, complex tables, native editable Word equations (OMML), and Mermaid timelines/flowcharts into clean.docxand vector PDF purely in the browser.
If you've ever finished a brutal on-call shift only to spend another hour fighting Word formatting or scrubbing logs by hand, I'd love your feedback on the live showcase: https://pathadd.com/en/tools/markdown2word/showcase.html
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.