База знаний » GNU/Linux

h1. Flashcache

*До fashcache*

<pre>

monkey ~ # fio -readonly -name iops -rw=randread -bs=4096 -runtime=20 -iodepth 1 -filename /dev/vda -ioengine libaio -direct=1

iops: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1

fio 2.0.7

Starting 1 process

Jobs: 1 (f=1): [r] [100.0% done] [482K/0K /s] [117 /0  iops] [eta 00m:00s]

iops: (groupid=0, jobs=1): err= 0: pid=2941

  read : io=8892.0KB, bw=455156 B/s, iops=111 , runt= 20005msec

    slat (usec): min=8 , max=312 , avg=35.66, stdev=21.26

    clat (usec): min=487 , max=68000 , avg=8957.66, stdev=4331.27

     lat (usec): min=528 , max=68015 , avg=8995.32, stdev=4332.05

    clat percentiles (usec):

     |  1.00th=[ 3440],  5.00th=[ 4512], 10.00th=[ 5024], 20.00th=[ 5920],

     | 30.00th=[ 6816], 40.00th=[ 7776], 50.00th=[ 8640], 60.00th=[ 9536],

     | 70.00th=[10432], 80.00th=[11200], 90.00th=[12096], 95.00th=[12864],

     | 99.00th=[27264], 99.50th=[36608], 99.90th=[52992], 99.95th=[60160],

     | 99.99th=[68096]

    bw (KB/s)  : min=  285, max=  503, per=99.99%, avg=443.95, stdev=49.29

    lat (usec) : 500=0.04%

    lat (msec) : 2=0.18%, 4=1.62%, 10=64.37%, 20=32.07%, 50=1.57%

    lat (msec) : 100=0.13%

  cpu          : usr=0.11%, sys=0.61%, ctx=2230, majf=0, minf=21

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=2223/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):

   READ: io=8892KB, aggrb=444KB/s, minb=444KB/s, maxb=444KB/s, mint=20005msec, maxt=20005msec

Disk stats (read/write):

  vda: ios=2197/2, merge=0/7, ticks=19630/103, in_queue=19728, util=98.76%

</pre>

*Холодный кэш*

<pre>

monkey ~ # fio -readonly -name iops -rw=randread -bs=4096 -runtime=20 -iodepth 1 -filename /dev/vda -ioengine libaio -direct=1

iops: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1

fio 2.0.7

Starting 1 process

Jobs: 1 (f=1): [r] [95.2% done] [452K/0K /s] [110 /0  iops] [eta 00m:01s]

iops: (groupid=0, jobs=1): err= 0: pid=1497

  read : io=8436.0KB, bw=431901 B/s, iops=105 , runt= 20001msec

    slat (usec): min=8 , max=70032 , avg=59.42, stdev=1524.52

    clat (usec): min=225 , max=133986 , avg=9419.83, stdev=5905.45

     lat (usec): min=289 , max=134035 , avg=9480.52, stdev=6126.76

    clat percentiles (msec):

     |  1.00th=[    3],  5.00th=[    5], 10.00th=[    6], 20.00th=[    7],

     | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[   10],

     | 70.00th=[   11], 80.00th=[   12], 90.00th=[   13], 95.00th=[   14],

     | 99.00th=[   28], 99.50th=[   43], 99.90th=[   64], 99.95th=[  126],

     | 99.99th=[  135]

    bw (KB/s)  : min=  246, max=  502, per=100.00%, avg=421.10, stdev=63.38

    lat (usec) : 250=0.09%, 500=0.33%, 750=0.05%

    lat (msec) : 2=0.05%, 4=1.38%, 10=61.21%, 20=34.90%, 50=1.71%

    lat (msec) : 100=0.19%, 250=0.09%

  cpu          : usr=0.07%, sys=0.76%, ctx=2116, majf=0, minf=21

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=2109/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):

   READ: io=8436KB, aggrb=421KB/s, minb=421KB/s, maxb=421KB/s, mint=20001msec, maxt=20001msec

Disk stats (read/write):

  vda: ios=2139/2, merge=1/4, ticks=20365/732, in_queue=21067, util=98.67%

</pre>

*Горячий кэш*

<pre>

monkey ~ # fio -readonly -name iops -rw=randread -bs=4096 -runtime=20 -iodepth 1 -filename /dev/vda -ioengine libaio -direct=1

iops: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1

fio 2.0.7

Starting 1 process

Jobs: 1 (f=1): [r] [100.0% done] [426K/0K /s] [104 /0  iops] [eta 00m:00s]

iops: (groupid=0, jobs=1): err= 0: pid=1502

  read : io=16912KB, bw=865634 B/s, iops=211 , runt= 20006msec

    slat (usec): min=7 , max=163 , avg=19.44, stdev=15.56

    clat (usec): min=172 , max=105321 , avg=4708.36, stdev=5534.23

     lat (usec): min=206 , max=105333 , avg=4728.79, stdev=5540.49

    clat percentiles (usec):

     |  1.00th=[  203],  5.00th=[  211], 10.00th=[  221], 20.00th=[  231],

     | 30.00th=[  241], 40.00th=[  258], 50.00th=[ 2384], 60.00th=[ 6176],

     | 70.00th=[ 7968], 80.00th=[ 9664], 90.00th=[11328], 95.00th=[12352],

     | 99.00th=[18816], 99.50th=[25216], 99.90th=[51968], 99.95th=[57088],

     | 99.99th=[104960]

    bw (KB/s)  : min=  302, max=13928, per=100.00%, avg=854.87, stdev=2191.98

    lat (usec) : 250=34.08%, 500=15.80%

    lat (msec) : 2=0.05%, 4=0.76%, 10=31.86%, 20=16.60%, 50=0.73%

    lat (msec) : 100=0.09%, 250=0.02%

  cpu          : usr=0.18%, sys=0.66%, ctx=4238, majf=0, minf=21

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=4228/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):

   READ: io=16912KB, aggrb=845KB/s, minb=845KB/s, maxb=845KB/s, mint=20006msec, maxt=20006msec

Disk stats (read/write):

  vda: ios=4205/2, merge=0/7, ticks=19509/68, in_queue=19577, util=98.09%

</pre>

*Сильно разогретый кэш*

<pre>

monkey ~ # fio -readonly -name iops -rw=randread -bs=4096 -runtime=20 -iodepth 1 -filename /dev/vda -ioengine libaio -direct=1

iops: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1

fio 2.0.7

Starting 1 process

Jobs: 1 (f=1): [r] [100.0% done] [469K/0K /s] [114 /0  iops] [eta 00m:00s]  

iops: (groupid=0, jobs=1): err= 0: pid=1543

  read : io=45572KB, bw=2278.5KB/s, iops=569 , runt= 20001msec

    slat (usec): min=8 , max=140 , avg=15.68, stdev=14.67

    clat (usec): min=134 , max=74829 , avg=1736.53, stdev=4006.16

     lat (usec): min=209 , max=74886 , avg=1752.89, stdev=4010.50

    clat percentiles (usec):

     |  1.00th=[  179],  5.00th=[  215], 10.00th=[  223], 20.00th=[  233],

     | 30.00th=[  237], 40.00th=[  241], 50.00th=[  247], 60.00th=[  251],

     | 70.00th=[  258], 80.00th=[  298], 90.00th=[ 8096], 95.00th=[10304],

     | 99.00th=[12864], 99.50th=[16320], 99.90th=[36608], 99.95th=[54016],

     | 99.99th=[72192]

    bw (KB/s)  : min=  328, max=15416, per=100.00%, avg=2324.00, stdev=4584.97

    lat (usec) : 250=56.80%, 500=26.99%

    lat (msec) : 2=0.03%, 4=0.25%, 10=10.30%, 20=5.27%, 50=0.31%

    lat (msec) : 100=0.05%

  cpu          : usr=0.33%, sys=1.20%, ctx=11412, majf=0, minf=21

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued    : total=r=11393/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):

   READ: io=45572KB, aggrb=2278KB/s, minb=2278KB/s, maxb=2278KB/s, mint=20001msec, maxt=20001msec

Disk stats (read/write):

  vda: ios=11388/2, merge=0/4, ticks=19568/315, in_queue=19882, util=97.67%

</pre>

*Статистика:*

<pre>

kvm320-monkey-a-cache: 0 41943040 flashcache stats: 

	reads(66944), writes(843)

	read hits(35892), read hit percent(53)

	replacement(0), write replacement(0)

	invalidates(2554)

	pending enqueues(0), pending inval(0)

	no room(0)

	disk reads(31053), disk writes(854) ssd reads(35892) ssd writes(23865)

	uncached reads(7188), uncached writes(854), uncached IO requeue(0)

	uncached sequential reads(0), uncached sequential writes(0)

	pid_adds(0), pid_dels(0), pid_drops(0) pid_expiry(0)

</pre>