Linux Disk & OS Drive

We’ve all seen it happen. An alert fires off at 2:00 AM: Disk Space Usage Critical: 100% on /.

The server is unresponsive. SSH drops. The application has completely locked up because the OS can't even write to /var/log/syslog. Why? Because someone installed a heavy, data-hungry application (like a database or a SIEM indexer) directly onto the root OS disk.

In production, separating your OS layer from your application data layer is not just best practice—it is survival.

Recently, while building out a Wazuh SIEM deployment on Azure, we had to attach a dedicated 256 GB Premium SSD purely for the Wazuh Indexer. When you attach a new disk to a Linux virtual machine, it doesn’t magically show up ready to use. You have to partition, format, mount, and persist it.

Here is a deep dive into how Linux storage actually works under the hood, using our Wazuh Indexer data disk as the real-world example.

---

Step 0: Reading the Block Devices (lsblk)

Before we touch anything, we need to see how the kernel currently perceives the hardware. We run lsblk (List Block Devices).

The Output:

• nvme0n1 → 128G (OS disk)

  • nvme0n1p1 mounted on /

• nvme0n2 → 256G (DATA disk)

What does this mean in depth?

In Linux, everything is a file, including your hardware. These disks live in the /dev/ (device) directory.

• nvme: Stands for Non-Volatile Memory Express. It tells us this is a high-speed SSD attached via the PCIe bus, not a legacy SATA drive (sda, sdb).

• 0n1 / 0n2: Controller 0, Namespace 1 (the 128G OS disk) and Namespace 2 (the attached 256G data disk).

• p1: Partition 1. Notice how the OS disk has p1 mounted to the root directory (/), but nvme0n2 has no partitions under it.

Our target is /dev/nvme0n2. It is raw, unpartitioned, unformatted, and completely unusable to the OS in its current state.

---

Step 1: Slicing the Disk (Partitioning)

A raw disk is just an empty warehouse. A partition table acts as the blueprint, telling the operating system exactly where the usable storage begins and ends. We use fdisk to draw these boundaries.

Bash

sudo fdisk /dev/nvme0n2

Inside the fdisk interactive prompt, we issue the following commands:

• n (New Partition): We tell fdisk we are creating a new boundary.

• p (Primary): We make it a primary partition. Historically, MBR (Master Boot Record) disks were limited to 4 primary partitions. Even on modern systems, p is standard for simple data disks.

• 1 (Partition Number): This is the first partition on the disk.

• ENTER (First Sector): We accept the default, starting the partition at the very beginning of the available space.

• ENTER (Last Sector): We accept the default, stretching the partition to the absolute end of the 256 GB disk.

• w (Write): This commits the blueprint. It writes the partition table to the disk header and exits.

If you run lsblk again, you will now see /dev/nvme0n2p1. The warehouse has walls, but it's still an empty concrete floor.

---

Step 2: Laying the File System (Formatting)

Linux cannot write files to raw sectors. It needs a filing system—a directory structure, a way to track metadata (permissions, timestamps), and a method to map file names to physical blocks on the SSD.

This is where we format the partition.

Bash

sudo mkfs.ext4 /dev/nvme0n2p1

Why ext4? The Fourth Extended Filesystem (ext4) is the battle-tested workhorse of the Linux world. When we run this command, mkfs (Make File System) builds:

1. Superblocks: The master record containing the total number of blocks, block size, and filesystem health.

2. Inodes: The index nodes. Every file in Linux gets an inode that stores its metadata.

3. The Journal: A circular log that tracks uncommitted writes. If the server loses power, the journal prevents massive data corruption by knowing exactly what was being written when the lights went out.

---

Step 3 & 4: The Mount Point (Attaching the Storage)

Unlike Windows, which assigns arbitrary drive letters (D:\, E:\), Linux uses a single unified directory tree starting at root (/). To use our new filesystem, we must graft it onto an existing branch of the tree.

First, we create an empty directory (the mount point):

Bash

sudo mkdir -p /var/lib/wazuh-indexer

Then, we bind the formatted partition to that directory:

Bash

sudo mount /dev/nvme0n2p1 /var/lib/wazuh-indexer

Now, whenever the Wazuh Indexer writes application data to /var/lib/wazuh-indexer, the Linux kernel intercepts those writes and physically places the data on the 256 GB nvme0n2 SSD, completely bypassing the 128 GB OS disk.

---

Step 5: Surviving a Reboot (/etc/fstab)

Here is where junior engineers bring down production. The mount command is temporary. It only exists in the kernel's RAM. If you reboot the VM right now, the disk will unmount, Wazuh will start up, find an empty directory, and either crash or start writing data to the OS disk.

We must make the mount permanent by editing the File System Table (fstab).

Bash

sudo nano /etc/fstab

We append this exact line:

Plaintext

/dev/nvme0n2p1 /var/lib/wazuh-indexer ext4 defaults,nofail 0 2

Decoding the fstab entry:

• defaults: Uses standard mount options (read/write, asynchronous I/O, etc.).

• nofail: Crucial for Cloud VMs. If Azure has a blip and the data disk is slow to attach during boot, nofail tells Linux to continue booting the OS anyway. Without this, the VM drops into an unreachable "emergency mode" and you are locked out.

• 0 (Dump): Tells the legacy dump backup utility to ignore this filesystem.

• 2 (Pass): Tells fsck (File System Check) to check the root disk (1) first during boot, and check this data disk (2) sequentially afterward.

---

Step 6: Verification and Permissions

We finalize the setup by verifying our work and setting baseline permissions.

Bash

df -h

(Disk Free - Human Readable). You should see /dev/nvme0n2p1 proudly displaying its 256 GB of capacity mounted exactly where we want it.

Finally, we ensure the directory is accessible for the upcoming installation:

Bash

sudo chmod 755 /var/lib/wazuh-indexer

(When we install Wazuh, it will automatically create its own service user and take chown ownership of this directory).

The Architectural Payoff

By doing this properly, we have guaranteed that even if the Wazuh Indexer spikes and ingests a massive wave of logs, the worst thing that can happen is the 256 GB data disk fills up. The OS disk remains untouched, the server stays online, SSH continues to function, and we have the breathing room to expand the disk in Azure without an outage.