Provisioning volumes and deploying a test application
After the successful deployment of DataCore Bolt within a cluster, to verify the basic functionality, create a PersistenVolume using a Bolt StorageClass. Deploy a Flexible IO (FIO) test pod to consume this volume. Use the FIO I/O test utility from within the pod to run I/Os on the volume.
It is advisable to deploy the application stack in a namespace other than DataCore Bolt's as the supportability tool bundles information about all the resources in DataCore Bolt's namespace.
Create a PersistentVolumeClaim (PVC)
Use kubectl to create a PVC based on a StorageClass that you created in Configuring DiskPools. Replace the value of the field storageClassName with the name of your own Bolt-based StorageClass.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: bolt-volume-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: INSERT_YOUR_STORAGECLASS_NAME_HEREDeploy the FIO Test Pod
Deploy an FIO test pod by applying the below YAML. The nodeSelector field in the specification ensures that the application that is the FIO pod is always scheduled on one of the storage nodes.
kind: Pod
apiVersion: v1
metadata:
name: fio
spec:
nodeSelector:
datacore.com/engine: io-engine
volumes:
- name: bolt-volume
persistentVolumeClaim:
claimName: bolt-volume-claim
containers:
- name: fio
image: nixery.dev/shell/fio
args:
- sleep
- "1000000"
volumeMounts:
- mountPath: "/volume"
name: bolt-volumeVerify the PersistentVolumeClaim (PVC)
To verify the PVC, run the command shown below. The status of the volume should be Bound.
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
bolt-volume-claim Bound pvc-fe1a5a16-ef70-4775-9eac-2f9c67b3cd5b 1Gi RWO bolt-1 15sVerify the Persistent Volume
To verify the PersistentVolume, run the command shown below.
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-fe1a5a16-ef70-4775-9eac-2f9c67b3cd5b 1Gi RWO Delete Bound default/ms-volume-claim mayastor-1 16mSubstitute the example volume name with that shown under the "VOLUME" heading of the output returned by the get pvc command, as executed in your own cluster.
Verify the Bolt Volume
To verify the Bolt Volume, run the command shown below. Bolt kubectl plugin is used to verify the status of the volumes.
ID REPLICAS TARGET-NODE ACCESSIBILITY STATUS SIZE
18e30e83-b106-4e0d-9fb6-2b04e761e18a 3 aks-agentpool-12194210-0 nvmf Online 1073741824 Verify the application
Verify that the pod is deployed successfully with the status "Running". It may take a few seconds after creating the pod before it reaches that status, proceeding via the ContainerCreating state.
Run the below command to verify the application.
Run the FIO Test Utility
The FIO Test Utility is executed against the PV for 60 seconds, checking that I/O is handled as expected and without errors. In this example, we use a pattern of random reads and writes, with a block size of 4K and a queue depth of 16.
kubectl exec -it fio -- fio --name=benchtest --size=800m --filename=/volume/test --direct=1 --rw=randrw --ioengine=libaio --bs=4k --iodepth=16 --numjobs=8 --time_based --runtime=60Example output
benchtest: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.20
Starting 1 process
benchtest: Laying out IO file (1 file / 800MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=376KiB/s,w=340KiB/s][r=94,w=85 IOPS][eta 00m:00s]
benchtest: (groupid=0, jobs=1): err= 0: pid=19: Thu Aug 27 20:31:49 2020
read: IOPS=679, BW=2720KiB/s (2785kB/s)(159MiB/60011msec)
slat (usec): min=6, max=19379, avg=33.91, stdev=270.47
clat (usec): min=2, max=270840, avg=9328.57, stdev=23276.01
lat (msec): min=2, max=270, avg= 9.37, stdev=23.29
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 4], 20.00th=[ 4],
| 30.00th=[ 4], 40.00th=[ 4], 50.00th=[ 4], 60.00th=[ 4],
| 70.00th=[ 4], 80.00th=[ 4], 90.00th=[ 7], 95.00th=[ 45],
| 99.00th=[ 136], 99.50th=[ 153], 99.90th=[ 165], 99.95th=[ 178],
| 99.99th=[ 213]
bw ( KiB/s): min= 184, max= 9968, per=100.00%, avg=2735.00, stdev=3795.59, samples=119
iops : min= 46, max= 2492, avg=683.60, stdev=948.92, samples=119
write: IOPS=678, BW=2713KiB/s (2778kB/s)(159MiB/60011msec); 0 zone resets
slat (usec): min=6, max=22191, avg=45.90, stdev=271.52
clat (usec): min=454, max=241225, avg=14143.39, stdev=34629.43
lat (msec): min=2, max=241, avg=14.19, stdev=34.65
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3],
| 30.00th=[ 3], 40.00th=[ 3], 50.00th=[ 3], 60.00th=[ 3],
| 70.00th=[ 3], 80.00th=[ 4], 90.00th=[ 22], 95.00th=[ 110],
| 99.00th=[ 155], 99.50th=[ 157], 99.90th=[ 169], 99.95th=[ 197],
| 99.99th=[ 228]
bw ( KiB/s): min= 303, max= 9904, per=100.00%, avg=2727.41, stdev=3808.95, samples=119
iops : min= 75, max= 2476, avg=681.69, stdev=952.25, samples=119
lat (usec) : 4=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.02%, 4=82.46%, 10=7.20%, 20=1.62%, 50=1.50%
lat (msec) : 100=2.58%, 250=4.60%, 500=0.01%
cpu : usr=1.19%, sys=3.28%, ctx=134029, majf=0, minf=17
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=40801,40696,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=2720KiB/s (2785kB/s), 2720KiB/s-2720KiB/s (2785kB/s-2785kB/s), io=159MiB (167MB), run=60011-60011msec
WRITE: bw=2713KiB/s (2778kB/s), 2713KiB/s-2713KiB/s (2778kB/s-2778kB/s), io=159MiB (167MB), run=60011-60011msec
Disk stats (read/write):
sdd: ios=40795/40692, merge=0/9, ticks=375308/568708, in_queue=891648, util=99.53%If no errors are reported in the output, then DataCore Bolt is configured correctly and is operating as expected. You may now create and consume additional PersistentVolumes with your applications.