Real-Time Streaming Analytics for Network Performance and Security Monitoring

By Rick Aguirre This week’s topic is real-time streaming analytics for network monitoring, specifically performance and security. The industry has gradually evolved from network performance monitoring based on a large amount of historical data displayed on charts to a model that performs Network Behavior Analysis (NBA) on real-time streaming data. With the vast amount of data in the network, the trick to perform real-time streaming analytics is the conversion of raw network data into metadata. This summary data can then be absorbed and understood by different analytic models in real-time. Today these models include comparative rule engines and machine learning. A new approach to real-time analysis is the use of a time series database where the information and usage in the network is stored based on a timestamp for each user, link, application and network element. Time series database technology and real-time streaming technology has rapidly expanded in the past few years. It provides an ideal platform for Network Behavior Analysis and the detection of unusual actions from normal operations. By establishing a baseline of network behavior in the time series database and streaming real-time data across these patterns, all users, links, network elements and applications can be monitored in real-time for security and performance issues. This information can include many different variables from network sources, packets, wire data, NetFlow, SNMP, Syslog, and others. Once the baseline is created, various techniques and models are utilized to determine unusual activity. If an anomaly is detected, a notification can be sent to someone in a network operations center (NOC) or security operations center (SOC) for investigation; simultaneously automated procedures can be invoked to gather the forensic data and/or proceed with an automated response. This automated response can include blocking malicious traffic, blocking ports, sending traffic to a honeypot or reconfiguring elements in the network. This technique is especially good for detecting the “unknown” including new malware and zero-day exploits. It is also extremely effective in detecting changes in bandwidth, protocol, and application use for network performance. Network Behavior Analysis coupled with signature-based detection system such as IDS/IPS, firewalls and other security products provide a comprehensive visibility fabric to ensure security and performance of the enterprise. The automated detection techniques embedded in this fabric and the associated automated response system can significantly improve the performance and reduce the burden on an IT department. Rick Aguirre is a veteran of the telecommunications industry. He has a successful record of developing start-up companies that have emerging, industry-changing technologies. As the founder of Cirries Technologies, he has led his team to develop the fastest data extraction and aggregation tools which deliver the right data at the right time for any application. Cirries’ products can digest data from multiple sources and reduce it to the right format for real-time notification, storage, or application use to reveal real-time performance and security of any network. Rick’s passion outside of work is youth sports. In addition to coaching his children’s teams, he has coached Lacrosse in under resourced communities and has served on the Board for the North Texas Chapter of Positive Coaching Alliance.