Powering network analytics with machine learning & AI

Artificial intelligence (AI) aims to create systems that emulate human intelligence. To achieve this, AI systems train on extensive datasets to develop models and algorithms capable of mimicking the human brain’s ability to learn from data, identify patterns and predict behaviors. Today, AI systems in various industries can autonomously perform complex tasks with unprecedented speed and accuracy, driving AI’s market value to USD 136.55 billion, with a projected CAGR of 37.3% from 2023 to 2030¹.

Machine learning (ML), which is often used synonymously with artificial intelligence, is a subset of AI that focuses on learning from existing data to identify or predict patterns. ML algorithms can establish acceptable patterns and detect deviations from the norm, aiding anomaly detection such as spam email recognition and fraud detection in financial transactions. Machine learning employs several techniques for data analysis and predictions, including supervised, unsupervised and reinforcement learning. Deep learning (DL) is a subset of machine learning that employs neural networks to recognize complex patterns hidden in humongous data sets, particularly in complicated areas such as image processing and natural language understanding. Deep learning algorithms enable extremely intricate use cases, such as autonomous vehicles that must perceive their surroundings and make real-time decisions.

The methodological fit: artificial intelligence in networking

IP networks are a methodological fit for AI because of the sheer volume of data that IP networks log every second. A network’s traffic reservoir provides adequate data to power any complex algorithm. Secondly, advancements in real-time traffic capture and processing through tools such as IP probes, ensure data completeness and zero-time lags. This guarantees both accuracy and relevancy across analytical outputs. Finally, the diversity of today’s traffic flows in terms of users, applications and services carries infinite information value, which when mined intelligently, can yield invaluable insights for operational improvements and business growth.

Next-gen DPI for network traffic analytics

All of these outcomes, however, require deep traffic visibility. ipoque’s OEM DPI engines, R&S®PACE 2 and R&S®vPACE, provide this visibility using several classification methodologies and a comprehensive signature library. ipoque’s traffic classification utilizes:

• Pattern matching which scans packet payloads
• Advanced behavioral, statistical and heuristic analyses which assess packet movements such as frequency and lags

Using these methodologies, DPI:

• Identifies protocols, application and service types
• Calculates critical insights regarding network usage and performance
• Establishes the network security posture by detecting malicious, anomalous and suspicious traffic

ML and DL algorithms address loss of visibility

Today, as much as 95% of all traffic flows are encrypted². Apps such as WhatsApp and Telegram use encryption. There is also an increasing use of anonymization and obfuscation techniques, such as VPN and TOR. This poses serious visibility issues for traditional DPI tools. For example, the latest encryption techniques, including TLS 1.3, ESNI and QUIC, progressively hide handshake information and some even conceal packet header data, rendering it impossible for traditional DPI tools to identify the underlying traffic flows.

Last year, ipoque introduced encrypted traffic intelligence (ETI) to address this limitation. ETI redrew the boundaries for DPI-based traffic inspection by combining:

• ML algorithms (e.g. k-nearest neighbors (k-NN) and decision tree learning)
• DL algorithms (e.g. convolutional neural networks (CNN), recurrent neural networks (RNN) and long short-term memory (LSTM) networks)
• High-dimensional data analysis
• Advanced caching methods

Boasting over 1,000 features including statistical, time series and packet-level features, these techniques enable real-time detection of the underlying applications. ETI therefore correlates visible data points relating to packet movements, accumulated over the long-term across vast traffic volumes. For instance, streaming traffic exhibits a consistent packet rate, while download traffic can show sudden bursts. Similarly, real-time communication apps like WhatsApp have specific packet rates and temporal patterns, such as low latency and sudden peaks.

Taking network analytics a notch higher with AI

Equipped with encrypted traffic intelligence, next-gen DPI solutions by ipoque are pushing network visibility to the next level. For example, ipoque’s R&S®PACE 2 and R&S®vPACE power a wide range of network traffic analysis tools, such as IP probes, traffic analyzers and network monitors, with accurate and reliable application classification information, regardless of encryption, obfuscation and anonymization. Full visibility enabled by ETI-enhanced DPI not only delivers superior traffic analytics in itself, but also significantly benefits downstream tools relying on these insights, namely networking and cybersecurity implementations such as switches, routers, gateways, VPN, APM, CASB, SWG, ZTNA, NAC, SSL Inspection and NGFWs. Fine-grained analytics from DPI are particularly crucial in supporting application-based rules programmed into such tools. Combining ETI-delivered traffic classification information with wider traffic data collected by DPI, for example performance metrics such as speeds, throughput and jitter enables networks to understand and analyze the impact of different encrypted and obfuscated traffic flows and respond to each accordingly.

For example, networks can uncover the latency for an encrypted application, such as the gaming app Roblox, or for a specific service within an encrypted application, such as WhatsApp Calling. ETI classifies each of these through subtle and complex behavioral patterns that are otherwise hard to detect through statistical analysis and heuristics alone. Networks can also cross aggregate selected services, such as video, across all applications to identify how well the network handles video traffic.

ETI’s fine-grained traffic insights also encompass users, device types and geographical locations, enabling networks to establish, for example, the QoE on the Teams Meeting app and the Zoom Conferencing app for users in a remote town. An encrypted application may perform differently on different smartphone OSS, for example iOS and Android, and on different device types, such as a laptop or a computer. Using these analytics, optimization and prioritization policies can be executed granularly by applications and user plans, despite encryption.

Uncovering hidden threats with AI

Encryption, unfortunately, has proven to aid attackers in concealing malicious activities, as in the case of malware such as ChromeLoader, Gamaredon, AdLoad, SolarMarker, and Manuscrypt³. In this case, encrypted traffic intelligence by ipoque correlates a wide range of traffic logs, for example periodic outbound traffic to remote C&C centers, to identify and inform tools such as CASB, IPS/IDS and DDoS prevention systems. This helps security tools establish threat analytics including share of malware traffic, resource loss, attack frequency and impact on network performance, alongside lists of compromised applications, users and devices. Similarly, irregularities within legitimate applications, for example peculiar use of ports, protocols, devices and locations, or abnormal transactions and timings can be used to expose phishing, data infiltration and ransomware activities, and related parameters.

A strategic tool for the future of network analytics

From energy grids to advanced smart cities, from space travel to military missions: data-dependent industry verticals need real-time network traffic analytics to manage the growing number of encrypted applications and services.

In this regard, ipoque has taken a major leap into the future by powering networks against the complexities arising from tougher encryption, obfuscation and anonymization techniques. ETI propels analytics to the forefront of network management, where accurate, fine-grained insights at application and service levels shape how data is created, processed, transferred and stored across millions of devices and machines around the globe.

With lightweight implementations, ipoque’s next-gen DPI equipped with ETI makes real-time network traffic intelligence available throughout the network, in both traditional and cloud environments. ETI also reinstates the role of DPI-based IP probes, traffic analyzers and application monitors while providing a great alternative to non-DPI techniques grappling with a gradual loss of traffic visibility due to increasingly aggressive encryption methods.

To learn more about encrypted traffic intelligence by ipoque, download our whitepaper: Encrypted traffic visibility.

Sources

[1] AI Market Report, GrandView Research - AI Market Report, GrandView Research
[2] Network Encryption: A Double-edged Sword for Cybersecurity
[3] Zscaler Blog - Zscaler Blog