linux-troubleshootinglisted

# Linux Troubleshooting Establish whether the host is the cause, using the four resources that actually run out: disk, memory, PIDs, and file descriptors. ## Purpose **Use this when:** - A host or Kubernetes node is degraded, unresponsive, or `NotReady`. - A systemd unit fails to start or restarts repeatedly. - An application fails with "no space left on device", "cannot allocate memory", "too many open files", or is killed with no log entry of its own. - A container symptom has no explanation inside the container. **Do not use this when:** - The host is healthy and the question is throughput or latency — use `linux-performance`. - The problem is connectivity — use `linux-network-debugging`. **What this skill assumes:** shell access to the host, and `sudo` for the parts that read other users' processes and the journal. State which you have; half of these commands silently show less without privileges. ## Operating Procedure 1. **Take a baseline in one pass.** Uptime, load, memory, disk. Four commands before any theory. 2. **Check the kernel's own record** — `dmesg` and the journal. The kernel logs OOM kills, disk errors and filesystem remounts that no application log will ever contain. 3. **Find what actually ran out.** Disk bytes, disk inodes, memory, PIDs, file descriptors. These fail differently and are confused constantly. 4. **Correlate with time.** When did it start? What changed then — a deploy, a cron job, a log rotation, a certificate expiry? 5. *