Why grep and URL Splitting Exposed Secrets 3 Times

📝 Originally published (in Japanese) at forge.workstyle.tech.

TL;DR I was investigating settings across multiple AI sessions and intended to keep secrets out of the output. URL splitting and grep filtering still left paths for passwords to reach the screen. I now check whether values can pass through before displaying output.

The takeaway

Splitting URLs or filtering lines with grep cannot eliminate every path by which secrets may remain in the output; database, Redis, and object storage passwords were displayed. I decided to redact anything that might contain secrets with Python, then review the output before publishing it to check for any remaining paths through which values could leak.

Why Splitting a URL Also Displayed the Database Password

In the initial output, I meant to extract only the host from a connection URL, but I ended up displaying the database administrator’s password too.

To extract just the host from the connection URL, I used this split:

v.split('://')[-1].split('/')[0]

The result kept the entire user:password@host portion, so the database administrator’s password was displayed too.

I wanted to extract only the host. But I hadn’t checked whether the result of splitting on delimiters included credentials. Even if the goal was “host only,” the code didn’t guarantee that.

This output was one reason to recreate the values. I recreated the database and Redis values during the same night, from 00:24 to 00:54.

Why Did grep -A4 Display the Redis Password?

The second time, while inspecting a configuration file, I used grep -A4 to show the four lines after the matching line, and it displayed the Redis password.

The plaintext Redis password on the adjacent line appeared in the output.

The records don’t show whether this happened before or after the rebuild. I can’t fill in the sequence. What remains is the result: when a secret is on a neighboring line, a search that displays surrounding lines can expose it too.

Why grep -v "value:" Still Revealed the Root Password

On the third attempt, at 00:4x, I ran grep -v "value:" intending to exclude lines containing values. The object storage root password still appeared.

The command to configure an alias for the object storage included the password as an argument. Because that line didn’t contain value:, it remained in the output, exposing the root password.

All three values had already been stored in plain text in the repository’s history. I rotated the storage root password at 01:02. Even though I had transformed the output to prevent exposure, there was still a path for the values under investigation to pass through.

Why od -c | sed Still Failed to Redact the Token

On October 10, I once again tried to hide a token with od -c | sed, but its prefix appeared in the conversation log.

od inserts spaces between characters, so the sed replacement didn’t match. As a result, the token’s prefix appeared in the conversation log.

I decided to ask for the token to be recreated. However, the record doesn’t say whether it was actually recreated. What we can confirm is only that I decided to request it.

The three incidents that happened three days later looked like different approaches, but followed the same pattern. After a step that exposed a value, they split strings or filtered out lines. They didn’t check whether the value remained in the processed result before outputting it.

How to Check for Secrets Before Outputting Results

From here on, I decided to redact anything that might contain secrets with Python before outputting it.

I don’t output the results of grep -A, grep -B, grep -C, cat, sed -n, git diff, or git show as-is.

When checking a URL, I use urllib.parse.urlsplit to extract only the hostname and port. When I need to compare values without revealing them, I use the beginning of their SHA-256 hashes. I also prepared a tool for this.

Using a tool doesn’t make the output safe automatically. Before outputting anything, I reread it to check whether there’s still a path for values to leak through.

It wasn’t enough to “write a command that hides secrets.” Each operation—splitting URLs, showing surrounding lines, and excluding lines by string—left a path for values to remain in the output.

Whether you’re investigating production settings or asking an AI agent to do the investigation, you can pause before reading the output. Check not the name of the redaction process, but whether there’s any path for values to make it into the output.

Key takeaways

• Use urllib.parse.urlsplit to extract only hostname and port from a URL.
• Redact output that may contain secrets with Python before displaying it.
• Do not display grep -A, grep -B, grep -C, cat, sed -n, git diff, or git show output directly.
• Before displaying output, check whether a value or a line containing it can still pass through.