Volume Quality of Service (QoS)

Explore this Page

• Overview
• Requirements
• QoS Parameters
• Creating a VolumeAttributesClass
• Applying QoS to a Volume
• Changing the QoS Policy of a Volume
• Verifying the Applied Limits
• Best Practices
• Benefits of Volume QoS

Overview

Volume QoS applies to Local PV LVM volumes only. It is not available for the other DataCore Puls8 storage engines.

You can limit the IOPS and bandwidth of a Local PV LVM volume by attaching a Kubernetes VolumeAttributesClass (VAC) to its PersistentVolumeClaim (PVC). A single StorageClass can then serve workloads with different performance profiles, and the limits on a volume can be changed later without recreating it or moving its data.

Limits apply to both filesystem and block-mode volumes, and are enforced on the node through the Linux cgroup v2 io.max interface.

By following the steps in this document, you will be able to create a VolumeAttributesClass with IOPS and bandwidth limits, apply it to a Local PV LVM volume, change or remove those limits on a running volume, and verify the limits that are actually in effect.

Requirements

To use volume QoS with Local PV LVM, ensure the following requirements are fulfilled:

• The cluster runs Kubernetes 1.34 or later. The VolumeAttributesClass API is not available in earlier versions.
• Every node that hosts these volumes runs cgroup v2.
• The node agent knows the kubelet directory. This defaults to /var/lib/kubelet/ and is set with lvmNode.kubeletDir where the kubelet root directory differs.

On a cluster earlier than Kubernetes 1.34, the cluster continues to operate normally but any VolumeAttributesClass configuration is ignored. No error is reported, and volumes run without the limits you expect. Confirm the Kubernetes version before relying on QoS limits.

QoS Parameters

Set the following parameters under parameters in a VolumeAttributesClass whose driverName is local.csi.openebs.io.

Setting a parameter to max removes the limit for that parameter. A value of 0 is not accepted.

The bandwidth parameters are spelled qosBandwith, without the letter d. Parameter names are case-sensitive and must not have leading or trailing spaces. A name that does not match exactly is rejected as an unsupported parameter rather than being ignored.

Choosing Unified or Directional Parameters

qosIopsLimit and qosBandwithPerSec are unified parameters that apply to both the read and the write direction. qosIopsReadLimit, qosIopsWriteLimit, qosBandwithReadPerSec and qosBandwithWritePerSec set each direction separately.

For each of IOPS and bandwidth, use either the unified parameter or the directional pair. If a unified parameter and one of its directional counterparts are both specified with different values, the VolumeAttributesClass is rejected and the limits already in effect on the volume are left unchanged.

Creating a VolumeAttributesClass

Define the QoS limits in a VolumeAttributesClass. The following example sets separate read and write limits.

apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: puls8-lvm-qos-standard
driverName: local.csi.openebs.io
parameters:
  qosIopsReadLimit: "10000"
  qosIopsWriteLimit: "5000"
  qosBandwithReadPerSec: "7000Mi"
  qosBandwithWritePerSec: "3500Mi"

Apply the configuration and confirm that the class is registered in the cluster:

kubectl apply -f vac.yaml
kubectl get volumeattributesclass

Applying QoS to a Volume

Reference the class from the PVC using the volumeAttributesClassName field.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lvm-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: puls8-lvm
  volumeAttributesClassName: puls8-lvm-qos-standard
  resources:
    requests:
      storage: 10Gi

Changing the QoS Policy of a Volume

Create a second VolumeAttributesClass with the new limits, then update volumeAttributesClassName on the PVC to reference it. DataCore Puls8 reconciles the new limits onto the existing volume automatically. The volume is not recreated and its data is preserved.

Only the parameters present in the new class are applied. A parameter that the new class does not specify keeps the value already in effect on the volume and is not reset to unlimited. To remove a limit, set that parameter to max in the new class rather than omitting it.

Verifying the Applied Limits

Confirm that the PVC references the expected class:

kubectl describe pvc <pvc-name>

The limits actually in effect are recorded on the LVMVolume custom resource for the volume, under spec.qos, as readIOPS, writeIOPS, readBPS and writeBPS. Bandwidth values are expressed in bytes per second.

kubectl get lvmvolume -n <namespace> <volume-name> -o jsonpath='{.spec.qos}'

Best Practices

• Define a small set of classes that describe workload tiers rather than one class for each application.
• Set IOPS and bandwidth limits together. A bandwidth limit on its own still permits a high rate of small I/O operations.
• Leave headroom above the measured steady-state demand of the workload so that recovery traffic is not throttled.
• Confirm the Kubernetes version and cgroup v2 support on every node before relying on these limits in production.

Benefits of Volume QoS

• A single StorageClass serves workloads with different performance requirements.
• Limits can be changed on a running volume without recreating it or moving data.
• A single workload is prevented from consuming the full device bandwidth of a node.