Essential Elements for Cloud Visibility

By Rick Aguirre This is the first in a series of posts on the various techniques for virtual network traffic visibility in the cloud. There are three critical tools for this visibility we will cover in this series: Data Capture, Data Brokering and Streaming Analytics. These tools can provide visualization and understanding of all the packets that travel in and out of virtual environments (North – South) and between applications (East – West) inside the cloud. In this post we focus on the most fundamental element of Data Capture, Raw Packet Capture. Packet Capture is the most complete source of understanding cloud data. Packet Capture enables comprehensive forensic investigation with the ability to drill through layers 2 — 7 of each packet. This data can be stored for examination and analysis to solve application/network performance issues and security breaches. In enterprise data center environments, Packet Capture has traditionally been performed by costly specialized hardware solutions. In the cloud, Packet Capture must have same flexible and scalable capability as the monitored applications. Cloud packet capture is enabled by a virtual tap off of a virtual switch or by embedding a software agent in the monitored applications virtual instance. An instance of packet capture must be able to be automatically deployed by the management and orchestration tools of the cloud provider. Typically, various traffic levels or CPU usage will trigger the deployment of additional instances.   In the cloud, the captured packet data will be stored for a period of time which can create challenges in retrieving targeted data. In the event of a traffic anomaly, it will be necessary to access the capture data from a central management point irrespective of the amount of data or the storage methodology. Finding essential data quickly can be facilitated by Metadata Capture. Metadata, in the form of Flow records (NetFlow, IPFIX) or Syslog records, can be acquired from applications, virtual switches and created from raw packets, which can then be fed to streaming analytics engines residing in the cloud. This near real-time analytic technology can be combined with a workflow function to automate the extraction of specific stored packets. Hardware-based Packet Capture has been an effective yet somewhat costly tool in the traditional enterprise data center. For cloud implementations, Packet Capture may become a more effective tool. Private data center personnel have access to many tools to diagnose applications and traffic performance; in a public cloud these tools today are limited. Because of this and the promising cost dynamics of the solution, Packet Capture could emerge as one of the top tools for performance and security monitoring in the cloud. Rick Aguirre is a veteran of the telecommunications industry. He has a successful record of developing start-up companies that have emerging, industry-changing technologies. As the founder of Cirries Technologies, he has led his team to develop the fastest data extraction and aggregation tools which deliver the right data at the right time for any application. Cirries’ products can digest data from multiple sources and reduce it to the right format for real-time notification, storage, or application use to reveal real-time performance and security of any network. Rick’s passion outside of work is youth sports. In addition to coaching his children’s teams, he has coached Lacrosse in under resourced communities and has served on the Board for the North Texas Chapter of Positive Coaching Alliance.