Skip to content

Artifact Analysis: Linux Process & Volatile Memory

Density:

In Linux, โ€œeverything is a file.โ€ The /proc directory is a pseudo-filesystem generated dynamically by the kernel. It does not exist on the hard drive; it resides entirely in memory.

Every running process has a dedicated subdirectory inside /proc named after its Process ID (PID) (e.g., /proc/1234/). By interrogating these directories, analysts can extract the exact state of an attackerโ€™s process.

  • cmdline: Contains the complete command line and arguments used to launch the process. Useful for spotting Base64 payloads or hidden flags.
  • cwd: A symbolic link to the Current Working Directory of the process. If a system daemon (like sshd) is working out of /dev/shm or /tmp, it is highly suspicious.
  • environ: Contains the environment variables loaded into the process. Attackers often leak stolen API keys or C2 configuration strings here.
  • fd/ (File Descriptors): A directory containing symbolic links to all files, named pipes, and network sockets opened by the process. It reveals exactly what the malware is communicating with.

The most valuable file in the PID directory is exe. This is a symbolic link to the actual binary on the disk. If an attacker drops a malware payload, executes it, and immediately deletes the physical file to hide from antivirus scans, the exe link will point to [path] (deleted). Finding an executing process mapped to a deleted binary is a massive, high-fidelity Indicator of Compromise (IOC).

Finding the malicious needle in a haystack of legitimate Linux processes requires behavioral analysis.

Just like in Windows Process Analysis, the parent-child relationship reveals the attackerโ€™s intent. Using the pstree -p -a command displays this hierarchy.

  • The Web Shell Signature: If a web server daemon (www-data running apache2, nginx, or php-fpm) spawns an interactive shell (bash, sh, dash) or a scripting engine (python, perl), the server has been compromised via a Web Shell or an RCE vulnerability.

Attackers frequently rename their binaries to blend in with normal system noise.

  • Typosquatting: Naming the malware systmend (instead of systemd) or kworker.
  • Space Padding: Naming the process "apache2 " (with a trailing space) to trick analysts skimming the top or ps output.
  • Kernel Thread Impersonation: Legitimate kernel threads are wrapped in brackets (e.g., [kthreadd]). Attackers may name their user-space malware [kworker/u4:2] to mimic a kernel thread. However, legitimate kernel threads have a PPID of 2 (kthreadd) and do not open network sockets. If a โ€œkernel threadโ€ has open TCP connections, it is an imposter.

A running payload is useless without Command & Control (C2). Analysts must bind suspicious network connections back to the process that initiated them.

Using modern tools like ss -lntp or lsof -i -P -n, analysts hunt for:

  1. Reverse Shells: An ESTABLISHED outbound connection to a public IP on an arbitrary port, initiated by a shell (bash, nc) or interpreter (python).
  2. Bind Shells: An unknown or suspicious process (e.g., perl) in a LISTEN state on a high, non-standard port (e.g., 4444, 31337).

If a sophisticated rootkit or in-memory implant is suspected, analysts must acquire a full RAM dump for offline analysis using tools like the Volatility Framework.

Unlike Windows, modern Linux kernels strictly prohibit reading /dev/mem for security reasons. Extracting RAM requires loading a specific Kernel Module.

LiME (Linux Memory Extractor)

The historical standard. It requires compiling a .ko (Kernel Object) module specific to the exact kernel version of the victim machine. This is highly reliable but operationally complex during a fast-moving incident.

AVML (Azure Voyager Memory Layout)

An open-source tool developed by Microsoft (written in Rust). It attempts to acquire memory without requiring custom kernel module compilation, bridging the gap between /proc/kcore and /dev/crash. It is highly recommended for modern cloud incident response.

When triaging a live, actively compromised Linux machine, deploy the following bash commands before initiating containment or shutdown procedures.

hunt_deleted_executables.sh
# Lists all running processes whose original binary has been deleted from disk.
# This immediately surfaces "fileless" malware and deleted droppers.
ls -l /proc/*/exe 2>/dev/null | grep "deleted"