Block Trace Analysis and Storage System Optimization

Appendix A: Tools and Functions

In this appendix, trace analysis tools based on MATLAB [85] and Python [86] are introduced. The functionalities are explained and sample code is provided. Although these two tools use different languages, they generally have common interfaces, such as the same input and output parameters.

MATLAB-Based Tool: MBPAR

The MATLAB-based Block-Trace Parser, Analyzer and Reporter ( MBPAR ) is designed for easy use. ¹ Without specified options, the default setting will do all the available analysis and output all the analyzed results into a Microsoft PowerPoint (ppt) file. The intermediate figures and data can be also saved into disk for next usage. Sample code is as follows:

% define the filename to parse and analyze

filename= D :\sample. t r c

% parse the blktrace file and translate the events into MATLAB matrix

blktrace_parser;

% analyze the IO events with/without raw report

batch_analysis;

% generate report in a given PowerPoint format

batch_generate_ppt;

Here the files blktrace parser.m , batch analysis.m , and batch generate ppt.m are all scripts. We use scripts for simplicity, although they can be easily converted into functions. For example, blktrace parser.m receives the filename of raw traces as input and output the parsed IO events as the input of batch analysis.m , which further output analyzed results and figures as input of batch generate ppt.m .

The output of blktrace parser.m mainly contains two variables: 1) lists_cmd is a N × 3 matrix, where N is the number of requests, and 3 columns represents the first LBA, the size, and the type (0 for write and 1 for read) of the request, respectively; 2) lists_action is a N × 2 matrix, which represents the arrival time and completion time of requests. Note that the input for the analysis functions can be from any type of traces other than blktrace, as long as the data format is the same for lists_cmd and lists_action .

The content of batch_analysis.m is straightforward. Essentially, it defines some data formats and calls all available analysis functions. There are some parameters (including these for subfunctions of analysis) to adjust the tool’s behavior:

options.offset time is used to adjust the starting time of the traces. In some traces, the starting time does not start from 0 so you need to find the starting time of the first event. The default value is 0.
options.time interval indicates the time window when calculating the average value in seconds. The default value is 1 second.
options.plot figure decides if the tool outputs a figure. When options.export report is true, options.plot figure will be set as true too.
options.plot fontsize regulates the font size used in the figure.
options.export report decides if a debug report with raw data and figures is generated. This option is only used in Windows system for debug purpose.
options.report name indicates the debug report name.
options.near_sequence configures whether a strict sequential stream (0) or a near sequential stream (1) is calculated.
options.lba_size_set adjusts the number of the LBA size set during some statistics. Each LBA range will be calculated as the total LBA/the number of sets.

The following is a sample configuration :

% batch_analysis.m

ptions.export_report=1;

if exist ('name','var')

options.report_name=[name, '_raw.ppt'];

else

options.report_name='trace_analysis_raw.ppt';

end

% report title

if options.export_report

saveppt2(options.report_name,'f',0,'t',...

[' Basic Workload Analysis Report'])

options.plot_figure=1;

end

options.plot_fontsize=10;

options.offset_time=0; [lists_action,idx]=sortrows(lists_action,1); lists_cmd=lists_cmd(idx,:);

After the simple setting, it starts to call all of the analysis functions. The analyzed data is saved for further usage, such as generating a PowerPoint analysis report.

%% call individual sub-functions

%0 get the very basic workload information

basic_info=sub_basic_info(lists_action,lists_cmd,options);

%1 average queue depth for completion and arrival

queue_record=sub_queue_depth(lists_action,lists_cmd,options);

%2 calculate the busy time of the device;

time_record=sub_busy_time(lists_action,options);

%3 average IOPS and throughput of requests options.time_interval=1; % set the time window = 1s average_record=sub_iops(lists_action,lists_cmd,options);

options.time_interval=6; % set the time window = 6s

average_record= sub_iops(lists_action,lists_cmd,options);

%4 calcuate the size distribution

req_size_record=sub_size_dist(lists_action,lists_cmd,options);

%5 calcuate the LBA/size distribution

options.lba_size_set=50; % adjust the range of LBA in plotting

lba_stat_array=sub_lba_dist(lists_action,lists_cmd,options);

%6 sequential analysis (stream/commands/size/queue length)

options.near_sequence=0; % sequential analysis

options.S2_threshold =32; % limit the minimum number which is counted as sequence stream

options.max_stream_length=1024;

options.seq_size_threshold=1024; % the size constraint for a sequential stream

sequence_stat=sub_sequence_analysis(lists_action,lists_cmd,options);

options.near_sequence=1; % near sequential analysis;

options.S2_threshold =32; % limit the minimum number which is counted as sequence stream

options.S2_threshold2 =64; options.max_stream_length=1024;

options.seq_size_threshold=1024; % the size constraint for a near sequential stream

sequence_stat=sub_sequence_analysis(lists_action,lists_cmd,options);

%7 calculate the sequence queue

% sub_sequence_queue(lists_cmd,options)

%8 stack distance analysis - WOW

% options.spec_stack=[10,20,30]; % specify the stack distance for very large datasets;

stack_wow_record=sub_stack_wow(lists_cmd,options);

%9 stack distance analysis - ROW

stack_row_record=sub_stack_row(lists_cmd,options);

%10 frequented write update ratio - WOW options.access_type=0; freq_wow_record=sub_freq_wow(lists_cmd,options);

%11 timed/ordered update ratio - WOW

options.access_type=0; time_wow_record=sub_time_wow(lists_cmd,options);

%12 seek distance calcuation

seek_dist_record=sub_seek_dist(lists_cmd,options);

%13 queue length and idle time

idle_queue_record=sub_idle_queue(lists_action,options);

save analyzed_data

batch generate ppt.m specifies the analysis contents. First, it needs a PowerPoint template, like workload.pptx . This template has some predefined frames/layouts with specified names via a master page. Here you use the tool named exportTopptx ² to access the slides:

options.ppt_template='E:\⎵workload.pptx'; exportToPPTX('open',options.ppt_template);

%% See all available masters and layout templates

% You can programmatically access master/layout information

% You can also run exportToPPTX by itself to have this information printed to command window

pptxInfo = exportToPPTX;

fprintf ('All⎵available⎵layout⎵templates:\n');

for ilayout=1:numel(pptxInfo.master(1).layout)

fprintf ('\t%d.⎵%s\n',ilayout,pptxInfo.master(1).layout(ilayout).name);

end

Second, it finds the corresponding analyzed results and figures from the saved dataset generated by batch analysis.m and creates the necessary remarks for the figures. Finally, it outputs a report in a pptx format. Take the IOPS page as an example. In this example, you use the predefined slide layout called 3Figure1Text, where there are some objects like Title 1, Footer Placeholder 2, Picture_11, Picture_12, Picture_22, and Text_main. These objects can be manipulated via the function exportToPPTX so you can easily adjust the content of the report based on requirements.

%% generate iops

filenames1= dir ('iops*.fig');

filenames2= dir ('throughput*.fig'); filenames3= dir ('reqsize*.fig');

% Layout #11: 3Figure1Text (Title 1, Footer Placeholder 2, Picture_11, Picture_12, Picture_22, Text_

a1= size (filenames1,1);

for i=1:a1

exportToPPTX('addslide','Layout','3Figure1Text');

exportToPPTX('addtext','Estimated IOPS and Throughput','Position','Title');

exportToPPTX('addtext','Basic Properties','Position','Text_sup');

for j=1:3

eval (['filenames=filenames', int2str (j),';']);

h = hgload(filenames(i).name);

set ( gcf , 'color', 'white');

set ( gca , 'color', 'none');

pic_pos=['Picture_' int2str (j)];

exportToPPTX('addpicture',h,'Position',pic_pos,'Scale','maxfixed');

close (h);

end

temp_str1='Observe⎵if⎵burst⎵and⎵idleness⎵exist';

temp_str2='Bursts⎵exist⎵if⎵there⎵are⎵peaks⎵much⎵higher⎵than⎵the⎵average';

temp_str3='Idleness⎵exist⎵if⎵there⎵are⎵troughs⎵much⎵lower⎵than⎵the⎵average';

A sample of the auto-generated presentation slide is shown in Figures A-1 and A-2 . The users can easily adjust the slide layouts for their scenarios.

Figure A-1

A generated PowerPoint sample report

Figure A-2

A generated PowerPoint sample report (cont’d)

Python-Based Tool: PBPAR

The Python-based tool is generally similar to the MATLAB-based tool . However, it provides more flexibility due to its wide interfaces to other programming languages. In addition, it is completed free and open sourced.

The following code shows how to call the analysis functions:

from numpy import *

from matplotlib.pylab import *

from scipy.stats import *

from scipy import *

import dill

from PBPAR import *

lists_action=lists_action[:,0:2]

options=options_class()

options.export_report = 0

options.report_name = 'trace_analysis.ppt' options.export_report = 0

options.plot_fontsize = 10

options.time_interval = 50

options.plot_figure = 1

options.offset_time = 0

idx=argsort(lists_action[:,0])

lists_cmd=lists_cmd[ix_(list(idx),[0,1,2])]

lists_action=lists_action[ix_(list(idx),[0,1])]

#0 obtain basic information

basic_info=sub_basic_info(lists_action,lists_cmd,options)

### call individual sub-functions

#1 average queue depth for completion and arrival

queue_record=sub_queue_depth(lists_action,lists_cmd,options)

#2 calculate the device busy time;

time_record=sub_busy_time(lists_action,options)

#3 average IOPS and throughput of requests

options.time_interval = 1

average_record=sub_iops(lists_action,lists_cmd,options)

options.time_interval = 6

average_record=sub_iops(lists_action,lists_cmd,options)

#4 calculate the size distribution req_size_record=sub_size_dist(lists_action,lists_cmd,options)

#5 calculate the LBA/ size distribution

options.lba_size_set = 50

lba_stat_array=sub_lba_dist(lists_action,lists_cmd,options)

#6 sequential analysis (stream/commands/ size /queue length )

options.near_sequence = 0

options.S2_threshold = 32

options.S2_threshold2 = 64

options.max_stream_length = 1024

options.seq_size_threshold = 1024 sequence_stat=sub_sequence_analysis(lists_action,lists_cmd,options)

options.near_sequence = 1

options.S2_threshold = 32

options.S2_threshold2 = 64

options.max_stream_length = 1024

options.seq_size_threshold = 1024 sequence_stat=sub_sequence_analysis(lists_action,lists_cmd,options)

#7 sequence queue analysis

## sub_sequence_queue(lists_cmd,options)

#8 stack distance analysis - WOW

# options.spec_stack=[10,20,30]; # for very large dataset stack_wow_record=sub_stack_wow(lists_cmd,options)

#9 stack distance analysis - ROW stack_row_record=sub_stack_row(lists_cmd,options)

#10 frequented write update ratio – WOW

options.access_type = 0 freq_wow_record=sub_freq_wow(lists_cmd,options)

#11 timed/ordered update ratio – WOW

options.access_type = 0 time_wow_record=sub_time_wow(lists_cmd,options)

#12 seek distance calcuation seek_dist_record=sub_seek_dist(lists_cmd,options)

#13 queue length and idle time idle_queue_record=sub_idle_queue(lists_action,options)

Finally, you can call batch generate ppt.py to create the slide similar to the MATLAB code. Note that you use the Python library python-pptx ³ as the wrapper to create the ppt file. python-pptx doesn’t need a PowerPoint installation.

Interaction Between MATLAB and Python

Due to the similarity, it is not very difficult to convert code between two languages. Here are some tools/links for reference ; Table A -1 compares them briefly:

Small Matlab to Python compiler (SMOP): Converts MATLAB/Octave code to Python code, https://github.com/victorlei/smop
LiberMate: A MATLAB-to-Python (SciPy/NumPy) translator, https://github.com/awesomebytes/libermate
OMPC: Open-source MATLAB-to-Python compiler, which is a bit outdated, although partially functional, ompc.juricap.com/

Table A-1

Comparison of Code Converters

Developers

First /

Latest

Required Library

Remarks

libermate

Eric C. Schug

March 2009/

May 2014

Numpy, scipy, matcompat

Imported libs are common

SMOP

Victor Lei

June 2013/

Dec 2016

smop

Supports Octave

OMPC

Peter Jurica

2008/

June 2010

ompc

Provides online version

Pylab of Python provides functions to read MATLAB mat files. Wrappers and interfaces between two languages are also available:

pymatlab: Interfaces and communicates with MATLAB from Python. Users can easily integrate a project with a large MATLAB codebase into Python scripts by using MATLAB scripts as a part of the Python program. https://pypi.python.org/pypi/pymatlab .
Python-Matlab wormholes : Allows both directions of interaction. However, only n-dimensional float arrays of data types are supported. https://github.com/pp5311006/python-matlab-wormholes .
Python-Matlab Bridge: Offers the matlab magic extension for iPython to execute normal MATLAB code from within iPython. Scipy is required to handle sparse arrays. https://github.com/arokem/python-matlab-bridge .
PyMat & pymat2: Allows Python programs to start, close, and communicate with a MATLAB engine session. The code is out of date. pymat.sourceforge.net/ , https://github.com/tinkuge/pymat2
mlabwrap, mlabwrap-purepy: Makes MATLAB look like a normal Python library using PyMat. mlabwrap.sourceforge.net/ .
pymex: Embeds the Python interpreter in MATLAB extension module. https://github.com/kw/pymex .
matpy: Calls Python from MATLAB . Users can access MATLAB in multiple ways, such as creating variables or manipulating .mat files. https://github.com/invenia/matpy .

Appendix B: Blktrace and Tools

Blktrace was developed by Jens Axboe in early 2000. Since Linux version 2.617-rc1, it has been embedded into the kernel. In fact, there have been no major changes since 2007. It has some main feature s as follows:

Low overhead, such as < 0.5% CPU usage of one core in Intel E5-1620
An easy-to-use configuration with simple CLI commands
Highly configurable with trace IO on one or several devices, plus user-selectable filter events
Low cost. Compared to a hardware bus analyzer, it’s free of charge.
Live and playback tracing

Figure B-1 shows the simplified structure of blktrace. You can see that it only considers device access after OS/FS cache. When IO enters to block IO layer (request queue), the relay channel per CPU gets events emitted, and blktrace then captures the events from the channels. Some events traces are listed as follows:

Q: Request queue entry allocated
S: Sleep during request queue allocation
I: Request queue insertion
M: Back merge of IO on request queue
F: Front merge of IO on request queue
T: Unplug due to timer
D: Request issued to underlying block dev
C: Request completed
P: Request queue plug operation
U: Request queue unplug operation
B: IO bounce operation
X: IO split operation
A: IO remap: MD or DM

Figure B-1

Blktrace structure

It has wide applications , e.g.,:

To analyze productivity of block devices (both real and virtual) and various hardware configurations
To calculate potential expenses of resources (for example, at connection of program RAID)
To define an optimal configuration for a specific program surrounding
To estimate productivity of various file systems (e.g., EXT4, JFS, XFS, Btrfs) via differently interaction with block subsystems
To analyze the efficiency of more complex systems, such as hybrid system, deduplication system, mobile storage system (e.g., Android), cloud system, etc.

Therefore, this tool has been extensively used in both academics and industries .

Industry: HP, Oracle, IBM, Intel, WDC, Seagate, Huawei, Taobao, DHT, SGI, etc.
Academics: Harvard University, University of California, Berkley, Imperial College of London, University of New South Wales, Florida International University, Sungkyunkwan University, Stony Brook University, University of Minnesota Twin Cities, Seoul National University, University of British Columbia, University of Maine, etc.

Some examples are listed as follows:

Alibaba/Taobao: Debug an broken output pipe problem in the new kernel 2.6.37 and monitor performance for HDFS, https://kernel.googlesource.com/pub/scm/linux/kernel/git/axboe/blktrace/9bf422b17cb2330f94376f8ca82a6e6cc496f9a3 .
IBM: Monitor and tune virtual I/O scheduler for virtualized storage systems, http://dl.acm.org/citation.cfm?id=1254826 .
Intel: Aid tool for Hystor (a high-performance hybrid storage system, SSD+HDD) design, http://dl.acm.org/citation.cfm?id=1995902 .
University of California, Berkeley: Determine the proper chunk size in HDFS for shared storage systems, www.eecs.berkeley.edu/alspaugh/papers/cake socc 2012.pdf .
University of Maine: Aid tool for migration algorithm design for hybrid storage systems, http://web.eece.maine.edu/jyue/papers/mascots11.pdf .

Blktrace has a simple user interface . Two typical usages are

Without command filter
$ blktrace -d /dev/sda -o blktrace.sda

With command filter

$ blktrace -d /dev/sda -a fs -o - |blkparse -i -

The results can be output to the terminal, HDD, RAM disk, or TMPFS. It may be better to store trace in a different device from the one traced. The command details can be found in its man page or you can refer to the user manual.

The trace collected by blktrace is in a binary format. In order to convert into a readable format, you need some tools to parse it. Blkparse produces formatted output of event streams of block devices. Figure B-2 provides an example output by blktrace. You can see that the IO request to device generally has D time (the time when request enters device) and C time (the time when request is completed), as well as some other file system-related time, such as queuing, insert, and merge.

Figure B-2

A sample trace collected by blktrace and parsed by blkparse

There are other tools related to blktrace:

Verify blkparse/blkrawverify verifies an output file from blkparse.
Btrace calls blktrace on the specified devices and pipes the output through blkparse for formatting.
Btt represents an abbreviation of the expression of the blktrace timeline that is possible to translate as the chronicle of blktrace, www.fis.unipr.it/doc/blktrace-1.0.1/btt.pdf .
Seekwatcher generates graphs from blktrace to visualize IO patterns and performance, https://oss.oracle.com/mason/seekwatcher/ .
Iowatcher graphs the results of a blktrace run, masoncoding.com/iowatcher/ .

These tools generally provide some basic information about the trace properties. For example, Seekwatcher visualizes some basic metrics such as throughput and IOPS, as shown in Figure B-3 . Iowatcher makes an animation of IO events. However, they do not provide any inside information related to cache and queue, which is particularly useful for the new generation disk drives, like SMR. This motivates the development of a dedicated tool for block-level trace analysis.

Figure B-3

Basic information by IOwatcher

Bibliography

[1]
R. Gastaldi and G. Campardo, Eds., In Search of the Next Memory: Inside the Circuitry from the Oldest to the Emerging Non-Volatile Memories . Springer, 2017.
[2]
M. Xie, L. Xia, and J. Xu, “State-dependent m/g/1/k queuing model for hard disk drives,” IEEE CASE (the 13th Conference on Automation Science and Engineering) , Xian, China, Mar 2017.
[3]
J. Niu, J. Xu, and L. Xie, “Optimal selection of garbage collection dirty threshold to balance power consumption and space release,” IEEE CAC , China, Oct 2017.
[4]
A. Amer, J. Holliday, D. Long, E. Miller, J. Paris, and T. Schwarz, “Data management and layout for shingled maganetic recording,” in IEEE trans. Magnetics , vol. 47, no. 10, pp. 3691–3697, 2011.
[5]
T. Feldman and G. Gibson, “Shingled magetic recording: areal density increase requires new data management,” USENIX; Login , vol. 38, no. 3, pp. 22–30, 2013.
[6]
J. Niu, M. Xie, J. Xu, L. Xie, and L. Xia, “Smr drive performance analysis under different workload environments,” Control Engineering Practice , vol. 75, pp. 86–97, 2018.
[7]
J. Niu, J. Xu, and L. Xie, “A deep look at smr performance via simulation approach,” IEEE ICCA (13th International conference on control and automation) , Ohrid, Macedonia, Jul 2017.
[8]
——, “Analytical modeling of smr drive under different workload environments,” IEEE ICCA (13th International conference on control and automation) , Ohrid, Macedonia, Jul 2017.
[9]
E. Brewer and L. Ying, “Disks for data centers,” File system and storage technology (FAST) , CA, USA, 2016.
[10]
Y. Shiroishi, K. Fukuda, I. Tagawa, H. Iwasaki, S. Takenoiri, H. Tanaka, H. Mutoh, and N. Yoshikawa, “Future options for hdd storage,” IEEE TRANS. MAGNETICS , vol. 45, no. 10, pp. 3816–3822, 2009.
[11]
C. F. Adams and C. R. McKie, “Balanced actuator which accesses separate disc assemblies,” Seagate, Patent US6115215A, 1998.
[12]
E. Kim, “Ssd performance - a primer: An intrduction to solid state drive performance, evaluation and test,” SNIA, Tech. Rep., 2013.
[13]
B. Schroeder and G. A. Gibson, “Disk failures in the real world: What does an mttf of 1,000,000 hours mean to you?” The 5th USENIX conference on File and Storage Technologies , Oct 2007.
[14]
J. Meza, Q. Wu, S. Kumar, and O. Mutlu, “A large-scale study of flash memory failures in the field,” ACM Sigmetrics , Portland, OR, USA, 2015.
[15]
J. Niu, J. Xu, and L. Xie, “Hybrid storage systems: A survey of architectures and algorithms,” IEEE Access , vol. 6, pp. 13 385–13 406, 2018.
[16]
J. R. David Reinsel, “Breaking the 15K-rpm HDD Performance Barrier with Solid State Hybrid Drives,” 2013.
[17]
M. K. Qureshi, V. Srinivasan, and J. A. Rivers, “Scalable high performance main memory system using phase-change memory technology,” ACM SIGARCH Computer Architecture News , vol. 37, no. 3, pp. 24–33, 2009.
[18]
Y. Huai, “Spin-transfer torque MRAM (STT-MRAM): Challenges and prospects,” AAPPS Bulletin , vol. 18, no. 6, pp. 33–40, 2008.
[19]
H.-S. P. Wong, H.-Y. Lee, S. Yu, Y.-S. Chen, Y. Wu, P.-S. Chen, B. Lee, F. T. Chen, and M.-J. Tsai, “Metal–oxide RRAM,” Proceedings of the IEEE , vol. 100, no. 6, pp. 1951–1970, 2012.
[20]
B. Jacob, S. W. Ng, and S. Rodriguez, Memory Systems: Cache, DRAM, Disk , Elsevier, 2008.
[21]
D. Lee, M. O’Sullivan, and C. Walker, “Measurement for improving the design of commodity archival storage tiers,” Utility and Cloud Computing (UCC), 2011 Fourth IEEE International Conference. IEEE, 2011, pp. 275–280.
[22]
I. Koltsidas, S. Sarafijanovic, M. Petermann, N. Haustein, H. Seipp, R. Haas, J. Jelitto, T. Weigold, E. Childers, D. Pease et al. , “Seamlessly integrating disk and tape in a multi-tiered distributed file system,” 2015 IEEE 31st International Conference on Data Engineering . IEEE, 2015, pp. 1328–1339.
[23]
N. Muppalaneni and K. Gopinath, “A multi-tier RAID storage system with RAID1 and RAID5,” Parallel and Distributed Processing Symposium, 2000. IPDPS 2000. Proceedings. 14th International . IEEE, 2000, pp. 663–671.
[24]
Wikipedia, “Flash memory — Wikipedia, the free encyclopedia,” 2017. [Online]. Available: “ https://en.wikipedia.org/wiki/Flash memory ”
[25]
H.-S. P. Wong, S. Raoux, S. Kim, J. Liang, J. P. Reifenberg, B. Rajendran, M. Asheghi, and K. E. Goodson, “Phase change memory,” Proceedings of the IEEE , vol. 98, no. 12, pp. 2201–2227, 2010.
[26]
K. A. Bailey, P. Hornyack, L. Ceze, S. D. Gribble, and H. M. Levy, “Exploring storage class memory with key value stores,” Proceedings of the 1st Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads , ACM, 2013, p. 4.
[27]
C. W. Smullen, J. Coffman, and S. Gurumurthi, “Accelerating enterprise solid-state disks with non-volatile merge caching,” Green Computing Conference, 2010 International , IEEE, 2010, pp. 203–214.
[28]
N. Lu, I.-S. Choi, S.-H. Ko, and S.-D. Kim, “A PRAM based block updating management for hybrid solid state disk,” IEICE Electronics Express , vol. 9, no. 4, pp. 320–325, 2012.
[29]
M. Tarihi, H. Asadi, A. Haghdoost, M. Arjomand, and H. Sarbazi-Azad, “A hybrid non-volatile cache design for solid-state drives using comprehensive I/O characterization,” IEEE Transactions on Computers , vol. 65, no. 6, pp. 1678–1691, 2016.
[30]
G. Sun, Y. Joo, Y. Chen, Y. Chen, and Y. Xie, “A hybrid solid-state storage architecture for the performance, energy consumption, and lifetime improvement,” Emerging Memory Technologies , Springer, 2014, pp. 51– 77.
[31]
W. Xiao, H. Dong, L. Ma, Z. Liu, and Q. Zhang, “HS-BAS: A hybrid storage system based on band awareness of Shingled Write Disk,” Computer Design (ICCD), 2016 IEEE 34th International Conference on , IEEE, 2016, pp. 64–71.
[32]
P. Petriuk, “Openstack architecture,” Mirantis, White paper, 2014.
[33]
S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn, “Ceph: A scalable, high-performance distributed file system,” Proceedings of the 7th Conference on Operating Systems Design and Implementation (OSDI 06) , 2006.
[34]
K. Singh, Learning Ceph . Packt Publishing, 2015.
[35]
D. G. Feitelson, Workload Modeling for Computer Systems Performance Evaluation , UK: Cambridge University Press, 2014.
[36]
R. Nou, J. Giralt, and T. Cortes, “Automatic i/o scheduler selection through online workload analysis,” in Ubiquitous Intelligence & Computing and 9th International Conference on Autonomic & Trusted Computing (UIC/ATC) , Fukuoka, Japan, 2012.
[37]
Y. Chen, “Workload-driven design and evaluation of large-scale data-centric systems,” PhD thesis, UC Berkeley, 2012. [Online]. Available: http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-73.pdf .
[38]
L. K. John and A. M. G. Maynard, Eds., Workload Characterization of Emerging Computer Applications , Springer Science+Business Media, LLC, 2001.
[39]
W. W. Hsu and A. J. Smith, “Characteristics of i/o traffic in personal computer and server workloads,” IBM Sysms Journal , vol. 42, no. 2, p. 347, 2003.
[40]
Q. M. Le, K. SathyanarayanaRaju, A. Amer, and J. Holliday, “Workload impact on shingle write disks: all-writes can be alright,” in 19th IEEE ISMASCTS , Jul 2011, pp. 444 – 446.
[41]
B. Gregg, Systems Performance: Enterprise and the Cloud , Prentice Hall, 2013.
[42]
A. Riska and E. Riedel, “Disk drive level workload characterization,” Annual Tech 06: USENIX Annual Technical Conference , 2006.
[43]
V. T. Priya Sehgal and E. Zadok, “Evaluating performance and energy in file system server workloads,” File system and storage technologies , 2010.
[44]
A. Williams, M. Arlitt, C. Williamson, and K. Barker, Web Information Systems Engineering and Internet Technologies Book Series , 2005, vol. 2, ch. WEBWORKLOAD CHARACTERIZATION: TEN YEARS LATER, pp. 3–21.
[45]
R. Eigenmann, Ed., Performance Evaluation and Benchmarking with Realistic Applications , The MIT Press, 2001.
[46]
S. Klyaus, Dynamic Tracing with DTrace & SystemTap , online book, http://myaut.github.io/dtrace-stap-book/ .
[47]
B. Gregg, DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X and FreeBSD , Oracle, 2011.
[48]
Wikipedia, “Cache replacement policies — Wikipedia, the free encyclopedia,” 2017.
[49]
L. Cherkasova, Improving WWW proxies performance with greedy-dual-size-frequency caching policy , Hewlett-Packard Laboratories, 1998.
[50]
E. J. O’neil, P. E. O’neil, and G. Weikum, “The lru-k page replacement algorithm for database disk buffering,” ACM SIGMOD Record , vol. 22, no. 2, pp. 297–306, 1993.
[51]
S. Jin and A. Bestavros, “Popularity-aware greedy dual-size web proxy caching algorithms,” Distributed computing systems, 2000. Proceedings. 20th international conference on , IEEE, 2000, pp. 254–261.
[52]
A. Traeger, E. Wright, N. Joukov, and C. P. Wright, “A nine year study of file system and storage benchmarking,” ACM trans. storage , vol. 4, no. 2, p. 5, 2009.
[53]
SPC, SPC BENCHMARK 1C , 1st ed., Storage Performance Council, CA, USA, Nov 2012.
[54]
J. S. Bucy, J. Schindler, S. W. Schlosser, and G. R. Ganger, The DiskSim Simulation environment version 4.0 reference manual , Carnegie Mellon University, PA, USA, 2008.
[55]
Futuremark, PCMark: the complete benchmark , 2nd ed., Futuremark, Apr 2016.
[56]
H. Kim, S. Seshadri, C. L. Dickey, and L. Chiu, “Evaluating phase change memory for enterprise storage systems: A study of caching and tiering approaches,” ACM Transactions on Storage (TOS) , vol. 10, no. 4, p. 15, 2014.
[57]
Y. Kim, A. Gupta, B. Urgaonkar, P. Berman, and A. Sivasubramaniam, “Hybridstore: A cost-efficient, high- performance storage system combining SSDs and HDDs,” in 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems , IEEE, 2011, pp. 227– 236.
[58]
D. L. Moal, Z. Bandic, and C. Guyot, “Shingled file system host-side management of shingled magnetic recording disks,” in Consumer Electronics (ICCE), 2012 IEEE International Conference on , 2012.
[59]
C. Jin, W.-Y. Xi, Z.-Y. Ching, F. Huo, and C.-T. Lim, “Hismrfs: a high performance file system for shingled storage array,” Mass Storage Systems and Technologies (MSST), 2014 30th Symposium on , Santa Clara, CA, USA, 2014.
[60]
C. Li, D. Feng, Y. Hua, and F. Wang, “Improving RAID performance using an endurable SSD cache,” Parallel Processing (ICPP), 2016 45th International Conference on , IEEE, 2016, pp. 396–405.
[61]
W. Liu, D. Feng, L. Zeng, and J. Chen, “Understanding the swd-based raid system,” in International Conference on Cloud Computing and Big Data , 2014.
[62]
A. Thomasian and Y. Tang, “Performance, reliability, and performability of a hybrid raid array and a comparison with traditional raid1 arrays,” Cluster Computing , vol. 15, pp. 239–253, 2012.
[63]
M. Li and J. Shu, “Daco: A high-performance disk architecture designed specially for large-scale erasure-coded storage systems,” IEEE TRANSACTIONS ON COMPUTERS , vol. 59, no. 10, pp. 1350–1362, 2010.
[64]
D. A. Patterson, G. Gibson, and R. H. Katz, “A case for redundant arrays of inexpensive disks (raid),” The 1988 ACM SIGMOD International Conference on Management of Data , 1988, pp. 109–116.
[65]
S. Wu, H. Jiang, L. Tian, and B. Mao, “Workout: Io workload outsourcing for boosting raid reconstruction performance,” 7th USENIX Conference on File and Storage Technologies, FAST 09 , San Francisco, CA, 2009.
[66]
Z. Ren, X. Xu, J. Wan, W. Shi, and M. Zhou, “Workload analysis, implications, and optimization on a production hadoop cluster: A case study on taobao,” Services Computing, IEEE Trans. on , vol. 7, no. 2, pp. 307 – 321, 2014.
[67]
G. Wang, A. R. Butt, H. Monti, and K. Gupta, “Towards synthesizing realistic workload traces for studying the hadoop ecosystem,” IEEE 19th MASCOTS , 2011, pp. 400–408.
[68]
Y. Chen, S. Alspaugh, and R. H. Katz, “Design insights for mapreduce from diverse production workloads,” UC Berkeley, http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-17.pdf , Tech. Rep. UCB/EECS-2012- 17, Jan 2012.
[69]
B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, and M. Paleczny, “Workload analysis of a large-scale key-value store,” SIGMETRICS ’12 , 2012, pp. 53–64.
[70]
J. Shafer, S. Rixner, and A. Cox, “The hadoop distributed filesystem: Balancing portability and performance,” Performance Analysis of Systems & Software (ISPASS) , Mar 2010, pp. 122 – 133.
[71]
S. Kavulya, J. Tany, R. Gandhi, and P. Narasimhan, “An analysis of traces from a production mapreduce cluster,” 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing , 2010, pp. 94–103.
[72]
C. L. Abad, N. Robert, Y. Lu, and R. H. Campbell, “A storage-centric analysis of mapreduce workloads: File popularity, temporal locality and arrival patterns,” IEEE International Symposium on Workload Characterization (IISWC) , 2012, pp. 100–109.
[73]
K. Ren, Y. Kwon, M. Balazinska, and B. Howe, “Hadoops adolescence: A comparative workload analysis from three research clusters,” Carnegie Mellon University, PA, USA, Tech. Rep. CMU-PDL-12-106, 2012.
[74]
G. Wang, A. R. Butt, P. Pandey, and K. Gupta, “A simulation approach to evaluating design decisions in mapreduce setups,” IEEE/ACM MASCOTS , London UK, Sep 2009.
[75]
J. Axboe, blktrace User Guide , 2007. [Online]. Available: www.cse.unsw.edu.au/~aaronc/iosched/doc/blktrace.html .
[76]
A. Brunelle, Block Layer tracing: blktrace , knfiu.mimuw.edu.pl, 2007.
[77]
T. Harter, D. Borthakur, S. Dong, A. S. Aiyer, L. Tang, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, “Analysis of hdfs under hbase: a facebook messages case study.” FAST , 2014, pp. 199–212.
[78]
C. Mason, Seekwatcher , Oracle, https://oss.oracle.com/mason/seekwatcher/ .
[79]
——, iowatcher , 2013. [Online]. Available: masoncoding.com/iowatcher/
[80]
S. A. Weil, “Ceph: Reliable, scalable, and high-performance distributed storage,” Ph.D. dissertation, University of California, Santa Cruz, CA, USA, 2007.
[81]
K. Singh, Ceph Cookbook . Packt Publishing, 2016.
[82]
D. Gudu, M. Hardt, and A. Streit, “Evaluating the performance and scalability of the ceph distributed storage system,” IEEE International Conference on Big Data , 2014, pp. 177–782.
[83]
X. Zhang, S. Gaddam, and A. Chronopoulos, “Ceph distributed file system benchmarks on an openstack cloud,” IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) , Bangalore India, Nov 2015, pp. 113–120.
[84]
F. Wang and M. Nelson, “Ceph parallel file system evaluation report,” National Center for Computatational Sciences, OAK RIDGE NATIONAL LABORATORY, ORNL/TM-2013/151, 2013.
[85]
S. Attaway, A Practical Introduction to Programming and Problem Solving using Matlab . Elsevier, 2013.
[86]
M. Luiz, Learning Python . Sebastopol, CA: O’Reilly Media, 2013.

Index

A

Apache Hadoop

B

Benchmark tools

PCMark

procedure

SPC-1C

storage performance

synthetic trace

trace collection

workload properties

Bit-patterned magnetic recording (BPMR)

Blktrace

academics and industries

applications

feature

IOwatcher

lists, events traces

structure

tools

trace collection

user interface

Block-level analysis, Hadoop

average size

average values of workloads

IOPS and Throughput

LBA and size distribution

LSD

observations and implication

queue depth

request size distribution

ROW ratio

sequence ratio with S=1024

sequence ratio with S=4096 and 8192

sequence stream

and command detection

with different N

trace statistics 148-21 (EXT4-WCD)

WCD vs . WCE

workload characteristics

workload, utilization

write update

Bottleneck identification, ceph

impacts, CPU, memory, network and disk

performance tool

software stack

C

Ceph

bottleneck identification

cluster topology

IO patterns

IOPS

LBA distribution

OSDs

verification, performance consistency

hypothesis approach

range tolerance approach

simplified statistical method

Ceph management, saltstack

Controlled Replication Under Scalable Hashing (CRUSH)

Conventional magnetic recording (CMR)

CRUSH algorithm

D

Devices, storage

common metrics

HDD ( see Hard disk drive (HDD))

hybrid disk

NVMs ( see Non-volatile memory (NVM))

SSD ( see Solid-state drive/disk (SSD))

tape and disc

DRAM protection

DTrace

E

Energy-assisted magnetic recording (EAMR)

Erasure code (EC)

F

Ferroelectric RAM (FeRAM)

G

GitHub

H

Hadoop

Cluster configuration

data flow

failure analysis

Google’s GFS

HDFS logs

LBA distribution

MapReduce framework

performance

SMR drives

trace collection

workflow

Hard disk drive (HDD)

areal density

BPMR technology

I

IO stack

J, K

Java Virtual Machine (JVM)

L

LBA distribution

Ceph node

Hadoop node

size

Logical Stack Distance (LSD)

Longitudinal magnetic recording (LMR)

LTTng

M

Magnetic tapes and optical discs

Magnetoresistive RAM (MRAM)

MapReduce system

MATLAB and Python

MATLAB-based Block-Trace Parser, Analyzer and Reporter (MBPAR)

batch analysis.m

batch generate ppt.m

blktrace parser.m

code

configuration

exportToPPTX

options

PowerPoint analysis report

PowerPoint sample report

Microservers and Ethernet drives

Microwave-assistant magnetic recording (MAMR)

N

Non-volatile memory (NVM)

flash with 3D techniques

Marketsandmarkets

NAND techniques

PCMS 3D Xpoint

ReRAM/CBRAM

SCM

STT-MRAM

SWOT analysis

types

O

Object storage daemons (OSDs)

Online transaction processing (OLTP)

OpenStack

P, Q

PBPAR

batch generate ppt.py

code

PCMark

applications

Cache prefetch

definition, holes

hole types

hypothesis

workload properties

completion time

size distribution

think/completion time (ms) and IOPS

think time distribution

Performance Test Specification (PTS)

Perpendicular magnetic recording (PMR)

Phase-change memory (PCM)

Physical block address (PBA)

Python

and MATLAB

PBPAR ( see PBPAR)

R

RAID and EC

hybrid systems

levels

SMEs

software-based implementations

Random access zone (RAZ)

Read-dominated trace

CDF

frequency update

idle time distribution

LBA distribution

segment

size distribution

SMR characteristics

stack update

time update

Read-modify-write (RMW)

Read on Write (ROW) ratio

Redundant array of independent/inexpensive disks (RAID)

data integrity requirements

OLTP systems

read-dominated trace ( see Read-dominated trace)

RMW accesses

system settings

technology

write-dominated trace ( see Write-dominated trace)

Resistive RAM (RRAM/ReRAM)

Response time, trace

S

SAS/SATA protocol

Seagate Technology

Sequence vs . randomness, workload

FIFO scheduling policy

LBA sequence

queued near-sequence stream

queued sequence stream with constraint

ratio

sequential stream

variables

Shingled magnetic recording (SMR)

algorithms

approaches

characteristics vs . workload metrics

conventional LBA-to-PBA mapping

drives

garbage collection (GC) procedure

HDD mechanics

high capacity

schematic of

Single large expensive disks (SLED)

Software-defined data center (SDDC)

Software-defined networking (SDN)

Software-defined storage (SDS)

common features

overall features

SDDC

SDN

TechTarget

Vmware

Webopedia

Wikipedia

Software trace collection

black box

Blktrace

DTrace

IO tools

LTTng

SAS/SATA protocol

static tools

SystemTap

Solid-state drive/disk (SSD)

flash memory

and HDD

market share

NVM

PCIe, SATA express, and M.2

performance tier

SAS and SATA

small size memory cards

SWOT analysis

trace analysis

block alignment

and HDD

IOPS vs. queue depth

NAND flash

NAND SSDs

NVM

sequential throughput vs . request size

SNIA

Solid-state hybrid drive (SSHD)

access isolation

cache size

flush/disk initialization commands

IOmeter traces

WCD

WCE

CMR models

NAND-backed DRAM cache protection

selection, models

SPC-1C tool

1K request trace

512K request trace

1024K request trace

vs. traditional HDDs

SPC-1C

decomposition

random walk pattern

spatial distribution

temporal distribution

workload ratio

Storage class memory (SCM)

Storage Performance Council (SPC)

Storage system

basics of storage

devices ( see Devices, storage)

implementation

Apache Hadoop

Ceph

OpenStack

microserver and Ethernet drives

performance evaluation

access patterns

common metrics

factors

IOPS

IO stack

redesign

response time (RT)

throughput (TP)

trace collection and analysis

vs. workload

RAID and EC ( see RAID and EC)

software-defined storage

Stream-detection algorithm

Synthetic trace

generator

inter-arrival time histogram

SPC-1C

System-level analysis, Hadoop

HDFS mechanism

HDFS metadata

JVM

MapReduce framework

System on a chip (SoC) architecture

SystemTap

T, U

Tape and disc

TechTarget

Toshiba’s KVDrive

Trace analysis

algorithms

hot data identification and data migration

performance factors

gain benefits, HDDs

HDD ( see Hard disk drive (HDD))

interactions

with applications

with structure

SSD ( see Solid-state drive/disk (SSD))

Trace collection

benchmark tools ( see Benchmark tools)

hardware method ( see Hardware trace collection)

software ( see Software trace collection)

techniques

warehouse

Trace metrics

advanced

mathematical models

priority-related

read/write dependency

sequential and near sequential streams

spatial locality and logical seek distance

statistical properties visualization

temporal locality

analytical model

basic

busy/idle time

estimation, queue depth and idle time

inter-arrival and inter-completion time

IOPS and throughput

LBA ( see LBA distribution)

mathematical models

queue depth

queue length

read and write distribution

response time

rotational positioning optimization

scheduling algorithms

typical workloads

VSCs

workload properties

file-level traces

numerical trace-based approach

typical application IO workload profiles

workload in system level

Two-dimensional magnetic recording (TDMR)

V

Vender special commands (VSCs)

Virtual machines (VMs)

Vmware

W, X, Y, Z

Warehouse, trace

Webopedia

Wikipedia

Workload and system performance

Workload metrics

block-level analysis ( see Block-level analysis, Hadoop)

system-level analysis ( see System-level analysis, Hadoop)

Workload properties

access patterns

application storage unit

business scaling unit

gain-loss analysis

cache ratio, hole filling policy

definition

hole filling

mixed simulation and modeling approach

prefetch and hole filling

prefetch policy

self-induced hole filling policy

simple method

HDD

IO pattern distribution

PCMark ( see PCMark)

size distribution

SPC-1C

Write cache disabled (WCD)

Write cache enabled (WCE)

Write-dominated trace

LBA distribution

size distribution

SMR characteristics

stack distance

stack update

Write once read many (WORM)

Write Update (Write on Write)

frequency

IO performance

ratio

stack distance

time period