EXT4 Quota

Explore this Page

• Overview
• Requirements
• Enabling EXT4 Quota
• Managing and Modifying EXT4 Quota
• Modifying the EXT4 Quota Limits
• Limitations

Overview

To enforce storage limits on Local PV Hostpath volumes held on an ext4 filesystem, enable and configure EXT4 project quotas. This document outlines the prerequisites, filesystem configuration, and StorageClass setup required to enable EXT4 quotas.

Additionally, it describes how to create Persistent Volume Claims (PVCs), verify quota enforcement, adjust quota limits when necessary, and remove a project quota from a volume.

By following the steps in this document, you will be able to enable, configure, and manage EXT4 Quotas for deployments across various Linux distributions, including Ubuntu, Debian, RHEL, CentOS, and Fedora.

EXT4 project quotas and XFS project quotas serve the same purpose for different filesystems. Use the configuration that matches the filesystem holding your BasePath.

Requirements

To use EXT4 Quotas with Local PV Hostpath, ensure the following requirements are fulfilled:

• Install the quota and e2fsprogs packages.
• Confirm that the filesystem type is ext4. If no ext4 filesystem is available, refer to Creating an EXT4 Filesystem on a Loop Device.
• Enable the project and quota features on the filesystem.
• Configure the filesystem to use the prjquota mount option.

Unlike XFS, an ext4 filesystem requires the project and quota features to be enabled before project quotas can be used. Filesystems created by older versions of mkfs.ext4 may not have them.

Installing the quota and e2fsprogs Packages

The quota package provides the repquota and setquota commands. The e2fsprogs package provides the tune2fs, chattr, and lsattr commands.

For Ubuntu/Debian Systems

sudo apt-get update
sudo apt-get install -y quota e2fsprogs

For RHEL/CentOS Systems

sudo yum install -y quota e2fsprogs

For Fedora Systems

sudo dnf install -y quota e2fsprogs

Checking the Filesystem Type

Verify whether the filesystem of the hostPath directory is ext4. The default hostPath directory is /var/openebs/local. Run the following command to check the filesystem type and identify the device that holds the filesystem:

df -Th /var/openebs/local

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/nvme1n1   ext4  8.0G  959M  7.1G  12% /mnt/data

If the command fails because the path does not exist yet, run the following script to determine the filesystem type and device name of the closest existing parent directory:

BASEPATH="/var/openebs/local"

until OUTPUT=$(df -Th $BASEPATH 2> /dev/null)
do
BASEPATH=$(echo "$BASEPATH" | sed 's|\(.*\)/.*|\1|')
done

echo "PATH=${BASEPATH}"
echo "$OUTPUT"

Enabling the Project and Quota Features

Enable the project and quota features on the device that holds the filesystem.

1. Check the features that are currently enabled on the device.

   Copy

   Check the Enabled Filesystem Features

   sudo tune2fs -l /dev/nvme1n1 | grep -i "filesystem features"

   Copy

   Sample Output

   Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum

   In the sample output above, project and quota are not listed, so you must enable them.

2. Unmount the filesystem. The features cannot be changed while the filesystem is mounted.

   Copy

   Unmount the Device (Replace with Your Device)

   sudo umount /dev/nvme1n1

3. Check the filesystem for errors. Do this before you change the filesystem features.

   Copy

   Check the Filesystem for Errors

   sudo e2fsck -f /dev/nvme1n1

4. Enable the project and quota features.

   Copy

   Enable the Project and Quota Features

   sudo tune2fs -O project,quota /dev/nvme1n1

5. Verify that the features are enabled.

   Copy

   Verify the Enabled Filesystem Features

   sudo tune2fs -l /dev/nvme1n1 | grep -i "filesystem features"

   Copy

   Sample Output

   Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum quota project

The project feature requires the filesystem to have an inode size of 256 bytes or more. This is the default for mkfs.ext4. A filesystem created with a smaller inode size cannot be converted and must be recreated. Check the inode size with sudo tune2fs -l /dev/nvme1n1 | grep -i "inode size".

Mounting the Filesystem with the prjquota Option

Checking Existing Mount Options

Verify whether the mount options for the device include prjquota.

sudo mount | grep "^/dev/nvme1n1"

/dev/nvme1n1 on /mnt/data type ext4 (rw,relatime)

If the mount options already include prjquota, proceed to Enabling EXT4 Quota. If not, continue with the steps for your filesystem.

Root Filesystem

If the filesystem is mounted as the root filesystem (/), enable prjquota through the GRUB configuration.

1. Edit the /etc/default/grub file.

   Copy

   Open /etc/default/grub

   sudo vi /etc/default/grub

2. Locate the line that contains the GRUB_CMDLINE_LINUX variable.

   Copy

   Locate GRUB_CMDLINE_LINUX Entry

   GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto net.ifnames=0 console=ttyS0"

3. Add rootflags=prjquota to the end of the string. If the rootflags option is already present, append prjquota to its list of options.

   Copy

   Add rootflags=prjquota

   GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto net.ifnames=0 console=ttyS0 rootflags=prjquota"

4. Locate the grub.cfg file. The path varies by operating system.

   Copy

   Possible Locations

   /boot/grub2/grub.cfg
   /boot/efi/EFI/ubuntu/grub.cfg
   /boot/efi/EFI/debian/grub.cfg
   /boot/efi/EFI/redhat/grub.cfg
   /boot/efi/EFI/centos/grub.cfg
   /boot/efi/EFI/fedora/grub.cfg

5. Create a backup copy of the existing grub.cfg file. The sample command uses the path /boot/grub2/grub.cfg. Replace it with the path for your system.

   Copy

   Backup Existing GRUB File

   sudo cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.backup

6. Generate a new grub.cfg file that includes the change.

   Copy

   Generate Updated GRUB Configuration

   sudo grub2-mkconfig -o /boot/grub2/grub.cfg

7. Reboot the system.

   Copy

   Reboot System

   sudo reboot

8. After the system restarts, check the mount options to confirm the change.

   Copy

   Check Mount Options

   sudo mount | grep " / "

   Copy

   Sample Output

   /dev/nvme0n1p1 on / type ext4 (rw,relatime,prjquota)

Filesystem on a Data Disk

If the filesystem is on a data disk, follow these steps. Replace /dev/nvme1n1 and /mnt/data with your device and mount path.

1. Unmount the filesystem on the data disk.

   Copy

   Unmount the Device (Replace with Your Device)

   sudo umount /dev/nvme1n1

2. Mount the disk with the prjquota mount option.

   Copy

   Mount the Device with prjquota

   sudo mount -o rw,prjquota /dev/nvme1n1 /mnt/data

3. Verify the mount options.

   Copy

   Verify prjquota Mount Option

   sudo mount | grep "^/dev/nvme1n1"

   Copy

   Sample Output

   /dev/nvme1n1 on /mnt/data type ext4 (rw,relatime,prjquota)

4. Add the prjquota option to the /etc/fstab file so that the change persists across reboots.

   Copy

   Add prjquota Option to /etc/fstab File

   UUID=9cff3d69-3769-4ad9-8460-9c54050583f9 /mnt/data               ext4     defaults,prjquota 0 0

Creating an EXT4 Filesystem on a Loop Device

If no existing device is formatted with an ext4 filesystem that has project quota enabled, you can create one on a loop device. This is useful when the root filesystem cannot be remounted with prjquota, as it allows you to evaluate project quota enforcement without repartitioning a disk.

The following steps create a sparse file, format it with the ext4 filesystem and the required features, and mount it as a loop device at /var/openebs/local.

1. Ensure that the quota and e2fsprogs packages are installed, as described in Installing the quota and e2fsprogs Packages.

2. Create the directory where the filesystem is mounted.

   Copy

   Create the Mount Directory

   sudo mkdir -p /var/openebs/local
   cd /var/openebs

3. Create a sparse file with a maximum size of 1 GiB. Use a size that can accommodate the volumes you intend to provision.

   Copy

   Create a 1 GiB Sparse File

   sudo dd if=/dev/zero of=ext4.1G bs=1 count=0 seek=1G

4. Format the sparse file with the ext4 filesystem, enabling the quota and project features that project quotas require.

   Copy

   Format the Sparse File with EXT4

   sudo mkfs.ext4 -F -O quota,project ext4.1G

5. Mount the sparse file as a loop device with project quota enabled. The file is then accessible as the /var/openebs/local directory.

   Copy

   Mount the Sparse File as a Loop Device

   sudo mount -o loop,rw,prjquota ext4.1G /var/openebs/local

6. Verify the mount options.

   Copy

   Verify prjquota Mount Option

   sudo mount | grep "/var/openebs/local"

   Copy

   Sample Output

   /var/openebs/ext4.1G on /var/openebs/local type ext4 (rw,relatime,prjquota)

A loop device backed by a sparse file is intended for evaluation and testing. Add the mount to /etc/fstab if it must persist across reboots.

Enabling EXT4 Quota

Complete the Requirements before you proceed. The filesystem that holds the BasePath must have the project and quota features enabled, and must be mounted with the prjquota option.

Creating a StorageClass

Create a hostpath StorageClass with the EXT4Quota configuration option. This enables EXT4 project quota for the specified base path and storage type.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: puls8-hostpath-ext4
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: "hostpath"
      - name: BasePath
        value: "/var/openebs/local/"
      - name: EXT4Quota
        enabled: "true"
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

Advanced EXT4 Quota Configuration

For advanced configuration, set the softLimitGrace and hardLimitGrace parameters. These define the storage capacity limits beyond the Persistent Volume (PV) storage request.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: puls8-hostpath-ext4
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: "hostpath"
      - name: BasePath
        value: "/var/openebs/local/"
      - name: EXT4Quota
        enabled: "true"
        data:
          softLimitGrace: "0%"
          hardLimitGrace: "0%"
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

To enable EXT4 Quota on the default hostpath StorageClass that the Helm chart creates, use the openebs.localpv-provisioner.hostpathClass.ext4Quota values, which accept enabled, softLimitGrace, and hardLimitGrace.

Creating a PVC

1. Create a PVC that uses the name of the StorageClass.

   Copy

   PVC using EXT4 Quota StorageClass

   kind: PersistentVolumeClaim
   apiVersion: v1
   metadata:
     name: local-hostpath-ext4
   spec:
     storageClassName: puls8-hostpath-ext4
     accessModes:
       - ReadWriteOnce
     resources:
       requests:
         storage: 5Gi

   At this stage, the PVC remains in the Pending state until the volume is mounted.

2. Verify the PVC status.

   Copy

   Verify PVC Status

   kubectl get pvc

   Copy

   Sample Output

   NAME                  STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS          AGE
   local-hostpath-ext4   Pending                                      puls8-hostpath-ext4   21s

Mounting the Volume

1. Mount the volume to the application pod container. The following sample uses a BusyBox pod.

   Copy

   Sample Pod Mounting the Volume

   apiVersion: v1
   kind: Pod
   metadata:
     name: busybox
   spec:
     volumes:
     - name: local-storage
       persistentVolumeClaim:
         claimName: local-hostpath-ext4
     containers:
     - name: busybox
       image: busybox
       command:
          - sh
          - -c
          - 'while true; do echo "`date` [`hostname`] Hello from Local PV Hostpath." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
       volumeMounts:
       - mountPath: /mnt/store
         name: local-storage

   The PVC status changes to Bound once the volume is mounted, and the quota is applied.

2. Verify that the EXT4 project quota is applied. Run the command on the node where the volume was provisioned.

   Copy

   Check EXT4 Quota Report

   sudo repquota -P /var/openebs/local

   Copy

   Sample Output

   *** Report for project quotas on device /dev/nvme1n1
   Block grace time: 7days; Inode grace time: 7days
                           Block limits                File limits
   Project         used    soft    hard  grace    used  soft  hard  grace
   ----------------------------------------------------------------------
   #0        --      20       0       0              2     0     0
   #1        --       0 5242880 5242880              1     0     0

3. Confirm the project ID that was assigned to the volume directory.

   Copy

   Check the Project ID of the Volume Directory

   sudo lsattr -pd /var/openebs/local/pvc-864a5ac8-dd3f-416b-9f4b-ffd7d285b425

   Copy

   Sample Output

       1 --------------P------ /var/openebs/local/pvc-864a5ac8-dd3f-416b-9f4b-ffd7d285b425

Managing and Modifying EXT4 Quota

Identifying the BasePath Directory

Make a note of the BasePath directory used for the hostpath volume. The default is /var/openebs/local. Retrieve the BasePath from the StorageClass with the following command:

kubectl describe sc <storageclass-name>

Locating the Node

Log in to the node where the volume exists. Determine the node by describing the Persistent Volume (PV) resource.

1. Retrieve the PVC details.

   Copy

   Get PVC Details

   kubectl get pvc --namespace demo

   Copy

   Sample Output

   NAME              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
   demo-vol-demo-0   Bound    pvc-0365904e-0add-45ec-9b4e-f4080929d6cd   2Gi        RWO            puls8-hostpath-ext4   21s

2. Describe the PV to find the node and the volume path.

   Copy

   Describe PV to find Node and Path

   kubectl describe pv pvc-0365904e-0add-45ec-9b4e-f4080929d6cd

   Copy

   Sample Output

   Name:              pvc-0365904e-0add-45ec-9b4e-f4080929d6cd
   Labels:            openebs.io/cas-type=local-hostpath
   Annotations:       pv.kubernetes.io/provisioned-by: openebs.io/local
   StorageClass:      puls8-hostpath-ext4
   Status:            Bound
   Claim:             demo/demo-vol-demo-0
   Reclaim Policy:    Delete
   Access Modes:      RWO
   VolumeMode:        Filesystem
   Capacity:          2Gi
   Node Affinity:
     Required Terms:
       Term 0:        kubernetes.io/hostname in [storage-node-2]
   Source:
       Type:  LocalVolume (a persistent volume backed by local storage on a node)
       Path:  /var/openebs/local/pvc-0365904e-0add-45ec-9b4e-f4080929d6cd

3. Identify the node name.

   Copy

   Get Node Information

   kubectl get node -l 'kubernetes.io/hostname in (storage-node-2)'

   Copy

   Sample Output

   NAME             STATUS   ROLES    AGE   VERSION
   storage-node-2   Ready    worker   10m   v1.32.4

Modifying the EXT4 Quota Limits

You can change the soft limit, the hard limit, or both, for an existing hostpath volume that has EXT4 project quota enabled. To remove the project quota entirely, refer to Removing the Project Quota.

Changing Quota Limits

Run the following commands on the node where the hostpath volume exists.

1. Make a note of the project ID.

   Copy

   Get Current Quota Report

   sudo repquota -P /var/openebs/local

   Copy

   Sample Output

   *** Report for project quotas on device /dev/nvme1n1
   Block grace time: 7days; Inode grace time: 7days
                           Block limits                File limits
   Project         used    soft    hard  grace    used  soft  hard  grace
   ----------------------------------------------------------------------
   #0        --      20       0       0              2     0     0
   #1        -- 1048576 2097152 2097152              1     0     0

   You can also read the project ID directly from the volume directory.

   Copy

   Read the Project ID from the Volume Directory

   sudo lsattr -pd /var/openebs/local/pvc-0365904e-0add-45ec-9b4e-f4080929d6cd

2. Modify the quota limits. The arguments are the project ID, the block soft limit, the block hard limit, the inode soft limit, and the inode hard limit, followed by the filesystem. The block limits are expressed in kilobytes.

   Copy

   Modify Quota Limits

   sudo setquota -P 1 3145728 5242880 0 0 /var/openebs/local

   The command above sets a soft limit of 3 GiB (3145728 KB) and a hard limit of 5 GiB (5242880 KB) for project ID 1. The inode limits are set to 0, which means they are unlimited.

3. Verify the updated limits.

   Copy

   Verify Updated Quota Limits

   sudo repquota -P /var/openebs/local

   Copy

   Sample Output

   *** Report for project quotas on device /dev/nvme1n1
   Block grace time: 7days; Inode grace time: 7days
                           Block limits                File limits
   Project         used    soft    hard  grace    used  soft  hard  grace
   ----------------------------------------------------------------------
   #0        --      20       0       0              2     0     0
   #1        -- 1048576 3145728 5242880              1     0     0

Removing the Project Quota

To remove the EXT4 project quota from a volume, follow these steps.

1. Make a note of the project ID.

   Copy

   Get Current Quota Report

   sudo repquota -P /var/openebs/local

2. Set the project limits to 0, which removes the quota limits.

   Copy

   Remove the Quota Limits

   sudo setquota -P 1 0 0 0 0 /var/openebs/local

   The command above applies to project ID 1 at the directory path /var/openebs/local.

3. Clear the project ID and the project inheritance attribute from the volume directory.

   Copy

   Clear the Project ID and Inheritance Attribute

   sudo chattr -P -p 0 /var/openebs/local/pvc-0365904e-0add-45ec-9b4e-f4080929d6cd

4. Verify the changes.

   Copy

   Verify the Quota Report

   sudo repquota -P /var/openebs/local

   Copy

   Sample Output

   *** Report for project quotas on device /dev/nvme1n1
   Block grace time: 7days; Inode grace time: 7days
                           Block limits                File limits
   Project         used    soft    hard  grace    used  soft  hard  grace
   ----------------------------------------------------------------------
   #0        -- 1048596       0       0              3     0     0

Limitations

Quota resizing is not supported: If you need to adjust quotas, you must manually modify the soft and hard limits as shown above.

Learn More

• XFS Quota
• StorageClass Parameters
• Creating a PVC