Network Excellence Rides on Streaming Analytics – And More

By Rick Aguirre

Streaming analytics has taken data analysis for business intelligence out of the backwater data warehouse and into the now. Organizations looking for new markets, clues to customer behavior, and other potentially actionable insights can get and analyze relevant data streams in just minutes, not days, weeks or even months after the fact.

 

But what about data needed for network behavior analysis? It’s been clear for some time that historical network monitoring technologies like SNMP or batch-based analytics are no longer enough. Today’s networks also require streaming analytics for data related to network behavior analysis. Even that dramatic improvement, however, needs something extra to master the exploding complexity of today’s networks and the growing tsunami of network behavior-related data running across networks today.

 

 

A database just for the network

 

As with the technology for streaming analytics, the technology behind the time series database has advanced rapidly over the past few years. Put simply, a time series database compiles and stores data that changes over time. This makes it especially appropriate for data about dynamic network conditions and events that are always shifting.

A time series database is precisely the database network managers and analysts need for effective and real-time network behavior analysis via streaming analytics. This type of database establishes a baseline historical record of network behavior. This record becomes the essential foundation for getting to know the network and for making all events across the network visible.

The next step is for network managers and analysts to use streaming analytics across these behavior patterns for monitoring all users, links, network elements, and applications in real-time.

 

Obviously, they cannot perform this by hand. Even reducing the stream to network related metadata only, there is just too much information moving far too fast for human brains to recognize and understand. This is where the other element needed for effective network behavior analysis fits right in.

 

The power of machine learning

 

In real estate, the mantra is location, location, and location. For today’s network behavior analysis, it’s machine learning. A truly comprehensive network behavior analysis package consists of streaming analytics, a time series database, and machine learning. Rising demand for security in network services is pushing all the network and Internet traffic toward encryption. In this situation, DPI techniques are no longer enough to detect anomalies using deterministic tools. Machine learning techniques, however, enable organizations to assess network traffic patterns and detect anomalies in network performance and QoE.

 

Machine learning provides more powerful insights about network behavior and possible problems because it identifies shifting patterns in huge data sets across multiple dimensions. Other metadata to include in any network behavior analysis are the WLAN control plane and APIs from other network functions, such as DHCP, firewalls, network management stations, and authentication servers.

This helps set up a model of the traffic data pattern, which network managers and analysts can then use to detect anomalies in network performance, application performance, and end users’ QoE. The goal, of course, is a faster MTTR whenever network issues arise.

 

 

Metrics and Metadata

 

And network concerns will crop up. It’s just the dynamic nature of ever more complex networks. That’s why it pays for every organization to think deliberately about the network performance metrics it most needs to monitor.

 

It’s also important to be prepared to change or adjust these metrics based on shifting internal or external organization circumstances or new business objectives. Network performance data metrics include but are by no means limited to:

 

     ·       Anomalies/intrusions.

·       Applications/devices.

·       Bandwidth.

·       Bottlenecks/delays.

·       Overall reliability.

 

Once IT has set up and refined the organization’s list of network behavior metrics, there’s another consideration. How quickly do the appropriate metadata reach the analytics applications? What if the needed metadata is not compatible with such applications? Or it is not available in as close to real time as possible?

 

It’s the nightmare scenario. Analysts or data engineers are wasting time and attention cleaning network behavior data manually. This causes delays in actual analysis that leave the network and possibly sensitive data wide open and vulnerable. If there is a network breach, a data hack, or any other network performance problem, network managers need metadata to analyze with as little delay as possible. They need metadata automatically formatted for analysis applications, not to mention automated alerts that help them focus on network concerns almost as soon as they arise.

 

 

The ROI on Network Excellence

 

Many business executives regard setting up and maintaining a network as purely a business expense to be minimized in every way possible. This mindset makes it harder for them to realize the true value to the organization of investing in network excellence tools like streaming analytics, coupled with a time series database and machine learning.

 

So, speak C-suite language to convince them. Talk about the business benefits and return on investment in behavior analysis tools that are updated for today’s and future 5G networks. The following are benefits to the organization of network excellence that any top-level executive will understand and support:

 

·       Top network performance.

·       More productive employees.

·       Happier customers/subscribers.

·       Increased data security and lower legal/regulatory risks.

 

How much will these tangible benefits cut any organization’s costs and boost revenues? That depends entirely on its individual situation. But it’s good to remind business decision makers that poor network performance can impact the organization’s financial results, its susceptibility to cyber threats, and even affect its reputation and future viability.

 

Network excellence also reduces the burden on IT. Long gone are the days of manually keeping the network secure and operating efficiently and sensitive data safe. Today’s networks are simply too complex, too loaded with multimedia traffic, and too tempting a target for cyber thieves to rely on outdated approaches to network behavior analysis. In addition, there is simply too much at stake in an era when most organizations depend on a network to settle for anything less than network excellence.