IOPerf results

Results were obtained using two sets of IO tests

Ll: 64 kByte buffer, 10000 IO
Ls: 4 kByte buffer, 80000 IO

All results are averaged over 10 passes. The test order in that table is :

IO test on individual partition (one at a time, no prefix)
One IO thread per volume, N volumes (1PV_)
Parralele IO, 10 threads per volume. One result-average shown. (NPV_)

Block IO Character Random

fwrite fread putc getc seek

Test tag & FS KB KB/sec Real KB/sec CPU %CPU KB/sec Real KB/sec CPU %CPU KB/sec Real KB/sec CPU %CPU KB/sec Real KB/sec CPU %CPU KB/sec Real KB/sec CPU %CPU

Ll__star_dataFC_test 320000 50879.71 204537.65 25.99 206727.46 237766.26 88.94 38528.02 44808.43 86.42 49936.79 50601.39 98.69 2655.35 2785.88 95.45

Ls__star_dataFC_test 320000 60320.86 198456.02 30.81 225273.65 235172.93 95.81 38860.55 43495.86 89.44 49879.64 50565.85 98.65 2704.97 2741.02 98.71

NPV_rcas6005Ll__star_dataFC_test 320000 7097.31 119042.59 5.98 4060.87 165427.70 2.49 4819.51 44270.75 10.89 4254.94 47315.38 8.98 1967.27 2492.42 79.12

NPV_rcas6005Ls__star_dataFC_test 320000 6254.49 128243.96 4.88 4672.36 161470.98 2.91 3989.39 43570.38 9.16 4405.53 47399.87 9.30 2095.29 2469.64 84.96

In the results above, there do not seem any major differences between 4 kBytes and 64 KBytes IO based tests. The multi thread tests show an almos linear scaling in write but a concerning factor of 50 less in block IO read.

Network aggregates

Further tests were performed from two networks. The results below

STAR CAS network	Reference

This test was performed by using maximum number of nodes from the CAS nodes and performing any of the test above (we already noted we did not see any major differences between small or larger chunks within 4 / 64 kBytes).
STAR's peak is at best 4 Gbits/sec (512 MByte/sec) while the reference plot (Phen-ix nodes) would indicate up to 6 Gbit/sec (768 MBytes/sec). We are 50% off peak performance and seem to be saturating. We currently attribute this discrepency to a network topology problem and investigating this.

The below graph gives a baseline comparing to the PANFS system which out-performed our previous NFS solutions.

At a maximum of 300 MB/sec at best, we would still exceed the previous solution performance. We also performed tests where both read and write would be ongoing simultaneously. The results could be seen below

IOzone profiling

The profiling gives a gross performance view for several kByte buffer and several file size.
The Read/Write tests are unbuffered while FRead/FWrite tests are buffered.

		Block IO						Character						Random
		fwrite			fread			putc			getc			seek
Test tag & FS	KB	KB/sec Real	KB/sec CPU	%CPU	KB/sec Real	KB/sec CPU	%CPU	KB/sec Real	KB/sec CPU	%CPU	KB/sec Real	KB/sec CPU	%CPU	KB/sec Real	KB/sec CPU	%CPU
Ll__star_dataFC_test	320000	50879.71	204537.65	25.99	206727.46	237766.26	88.94	38528.02	44808.43	86.42	49936.79	50601.39	98.69	2655.35	2785.88	95.45
Ls__star_dataFC_test	320000	60320.86	198456.02	30.81	225273.65	235172.93	95.81	38860.55	43495.86	89.44	49879.64	50565.85	98.65	2704.97	2741.02	98.71
NPV_rcas6005Ll__star_dataFC_test	320000	7097.31	119042.59	5.98	4060.87	165427.70	2.49	4819.51	44270.75	10.89	4254.94	47315.38	8.98	1967.27	2492.42	79.12
NPV_rcas6005Ls__star_dataFC_test	320000	6254.49	128243.96	4.88	4672.36	161470.98	2.91	3989.39	43570.38	9.16	4405.53	47399.87	9.30	2095.29	2469.64	84.96