Machine learning (ML) is a hot buzzword. Every business and network analytical tool is looking for ways to differentiate its offering, usually by claiming that more intelligence has been embedded in it. A great deal of this is just hype. Decades of traditional log analysis and visualization tool development has not instantly been replaced by Skynet or Colossus, two intelligent computer systems described in sci-fi movies. Dashboards are UI, not artificial intelligence, and in many cases, no dynamic learning based on ML techniques is required, or even useful.
However, machine learning tools can be applied to network analytics. This is a broad term for how we attempt to place differing network behaviors into relevant classes. There are many benefits of machine learning for analytics; one of the most important is that smart people are expensive and easily bored. Since they are often analyzing an unending stream of applications, network health and threat assessments daily, machine learning is the best tool for providing relief by replicating human experience into actionable software.
Since machine learning can be applied to all three areas, it’s easy to see that the future of analytics is firmly on the road to increasing levels of automated pattern analysis and ML-based behavior prediction. Machine learning algorithms have matured, especially supervised training of deep neural networks, and when paired with powerful vector processing (largely GPU/TPU based cloud computing) can meet or exceed human performance for any classification problem that can be outlined.
Here are three ways that machine learning is becoming a game-changer:
1) Network Health
Network health is one of the most straightforward and useful areas for ML. Network segments and components can suffer from any number of issues, from memory faults and hardware problems such as overheating to noisy signals due to component decays. Since switches are full multicore computers, they have the usual problems of deadlocks, memory leaks, bed updates, thrashing, crashing, etc. Network configuration and traffic variations result in all manner of undesirable conditions including unacceptable latencies, throughput, packet loss/retries, failures to meet QoS thresholds and congestion. Detecting all of these issues early is key to fixing them before the network experiences an actual outage.
Conducting a log analysis of network health is tricky and time consuming. SNMP network monitors emerged in the 1980s as system analysis tools, but today, these are often overwhelmed by the sheer complexity and dynamics of modern networks. Here’s where ML becomes most useful. Simple threshold-based alarms and monitors are replaced by ML modules that detect problems and trigger alerts, usually with significantly lower false positive rates than their static counterparts.
Network activity visualization is another area where unsupervised ML can be useful. When coupled with tools such as Tableau, interesting patterns can be classified by relatively low cost vector quantification algorithms such as k-means clustering. With these tools, there’s always the risk of generating ridiculograms (fancy-looking but useless diagrams), but often they can be used to spot trends and predict future problems that would otherwise be impossible to spot. Effective capacity planning is only possible when you can see how closely the system is to overloading. Predictive models provide “what if” stress test views that estimate the loads of holiday, disaster or failover traffic.
2) Application Monitoring
The bottom line question for conducting most analyses is, “How well are my applications running?” This is the reason institutions use networks in the first place, so analytics that create an understanding of site usage, response times and more have direct corollaries with customer satisfaction. In network analytics, the same tools that ensure network health are readily extended to monitor application specifics.
Clickstreams, video, audio and application logs provide the primary inputs for training application ML modules. Unlike network health, application monitoring often involves bespoke programming to classify the specific issues important to a given site. Instrumentation is also less standardized, resulting in a wider set of implementation issues.
3) Security, Threat Administration and Remediation
External and internal network and application attacks from malicious individuals and organizations top the list of concerns addressed by ML-based network analytics. Firewalls and active anti-malware tools remain the first line of defense against hackers, but analytics provide the basis for detecting, isolating, and at least partially remediating these issues.