Therefore it would be necessary to process the complete pcap file by each instance to extract all streams to identify which streams of the file are to be analyzed by the concrete worker instance. The format consists of a binary representation of the data which is structured by the time of the captured packets and not by logical packet traces. Applying this approach to the analysis of pcap files, the structure of the pcap format carries some inherent problems. This could be done by using multiple instances or by using frameworks like Amazon MapReduce which are designed to process huge sets of data. The overall performance of the test was seemingly limited by the I/O performance restriction within virtualized instances and virtualized storage systems. Following the overall cloud computing paradigm, performance limitations of this kind might be circumvented by using multiple resources which do the processing in parallel. The only requirement (for our tests) is the existence of a sufficient number of EBS volumes which then contain parts of the pcap file to be analyzed.ĭuring the benchmarks, the performance was significantly lower than with the setup described in the previous post, even though eight different EBS volumes were used to avoid the bottleneck of a single storage volume. Therefore the amount of storage can be scaled up very easily.

EBS volumes can be created and attached issuing two commands of the amazon ec2 command line tools. Each EBS volume is hosted within a specific availability zone and can be attached to instances running in the same zone. Since the I/O performance of single disks had turned out to be the bottleneck in the “local” setup, eight Elastic Block Storage (short: EBS) volumes were created and attached to the instance. According to Amazon, the technical specs are the following:Ĩ EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each) For our lab setup, a so-called extra large instance was used. The Amazon Elastic Compute Cloud (short: EC2) provides a flexible environment for the on demand provisioning of virtual machines of different performance levels. We’ll provide the results from some additional tests supported by public cloud services, namely AWS (Amazon Web Services). This is the third (and last) part of the series (parts 1 & 2 here).

Sample pcap files for analysis