PcapWT: An Efficient Packet Extraction Tool for Large Volume Network Traces
Young-Hwan Kim, Roberto Konow, Diego Dujovne, Thierry Turletti,
Walid Dabbous, and Gonzalo Navarro,
Network packet tracing has been used for many different purposes during
the last few decades, such as network software debugging, networking
performance analysis, forensic investigation, and so on. Meanwhile, the size of
packet traces becomes larger, as the speed of network rapidly increases. Thus,
to handle huge amounts of traces, we need not only more hardware resources,
but also efficient software tools. However, traditional tools are inefficient
dealing with such big packet traces. In this paper, we propose pcapWT, an
efficient packet extraction tool for large traces. PcapWT provides fast packet
lookup by indexing an original trace using a Wavelet Tree structure. In
addition, pcapWT supports multi-threading for avoiding synchronous I/O and
blocking system calls used for file processing, and is particularly efficient
machines with SSD. PcapWT shows remarkable performance enhancements
in comparison with traditional tools such as tcpdump and most recent tools
such as pcapIndex in terms of index data size and packet extraction time.
Our benchmark using large and complex traces shows that pcapWT reduces
the index data size down below 1% of the volume of the original traces.
Moreover, packet extraction performance is 20% better than with pcapIn-
dex. Furthermore, when a small amount of packets are retrieved, pcapWT is
hundreds of times faster than tcpdump.