DPI conquers encrypted traffic with machine learning and deep learning

Encrypted traffic is becoming increasingly widespread as concerns over security and privacy continue to grow. According to research by FortiGuard Labs¹, more than 85 % of global network traffic is encrypted. Popular encryption protocols include TLS (Transport Layer Security), SSL (Secure Sockets Layer), SSH, PGP and IPSec, which leverage both symmetric and asymmetric encryption methods such as RSA and AES.

Encryption has deep implications for the delivery of applications and network management. It protects critical applications, applications handling sensitive information and applications that are susceptible to interception via techniques such as sniffing. In addition, encryption provides companies and individuals with highly secure means of communication. And although encryption keeps packets obscure and safe, it poses various visibility and monitoring challenges for network operators. They can no longer identify cyberattacks, or implement traffic management policies such as SLA-based routing, application-specific content caching and content-specific optimization as the underlying protocols, applications and application attributes remain concealed.

Machine learning and deep learning for encrypted traffic visibility

Prior to the days of encryption, network operators deployed probes, deep packet inspection (DPI) engines and other traffic monitoring and traffic management tools to capture and process network traffic. An advanced DPI engine such as R&S®PACE 2 uses pattern matching, behavioral and statistical/heuristic analysis to identify traffic by applications and protocols, delivering traffic information in real-time to support various traffic management policies and safeguard networks from cyberthreats. With encrypted traffic, however, traditional DPI methods are no longer able to identify and deliver the required levels of traffic visibility, resulting in the need for more advanced DPI methods capable of analyzing encrypted traffic. This is where machine learning (ML) and deep learning (DL) come in to enhance our DPI software with encrypted traffic visibility.

What is machine learning and deep learning, exactly?

Machine learning (ML) uses algorithms built around ‘features’, which are essential parameters of significance relating to a data set. With these algorithms, machines then learn from a given set of examples to produce ‘intelligent’ outputs. In network management, ML could consist of algorithms with features such as user types, network attributes such as speeds, jitter and throughput and traffic information such as packet size or source address. The use of ML requires deep knowledge about the field of application. That means network managers have to ensure that the algorithms are developed by experienced experts with deep know-how in areas of network management. ML depends on the correct identification of features and the correct translation of the algorithms.

While machine learning has enabled operators to automate traffic and network management – from routing traffic to resolving network issues to optimizing network resources – it still hinges on human expertise. Deep learning (DL), however, leverages huge data sets available from the network to automatically identify features that define the state of the network. It deploys layered processing of data, which mimics the human brain’s way of analyzing information. This leads to an accurate identification of features, around which algorithms are built. As networks continue to grow and more data is collected across network management tools, operators’ DL capabilities increase with more accurate features and algorithms. Operators are then able to better monitor network performance, diagnose issues, manage traffic flows and predict network behavior.

Analyzing encrypted traffic with machine learning and deep learning

The incorporation of machine learning and deep learning in our next-gen DPI software allows for Encrypted Traffic Intelligence (ETI). ETI delivers encrypted traffic visibility and tackles the loss of traffic visibility experienced by traditional DPI methodologies. Combined with pattern matching, behavioral analysis and statistical/heuristic analysis, ML and DL, with their optimized algorithms and models for real-time processing, leverage operators’ ever-expanding data sets to identify features that can be used to detect the underlying traffic. Possible features include known denominators such as specific signature patterns or attributes such as latency or a flow’s entropy. Features can also be previously unknown to current tools such as a distinct flow pattern indicative of certain malware or a specific flow entropy indicative of an application type.

The superior intelligence introduced by ML and DL enables our DPI software to address various developments in encrypted traffic. This includes the introduction of the latest encryption methods such as TLS 1.3, TLS 1.3 0-RTT, ESNI, DNS over TLS and DNS over HTTPS. The capabilities introduced by ML and DL not only address challenges surrounding the identification of encrypted traffic, they also deliver real-time traffic and application intelligence for general traffic, which complements policy control engines, network packet brokers and security applications such as firewalls, web filtering and ATP systems. Especially in security applications, a mix of advanced machine learning algorithms combined with different deep learning layers is crucial in identifying novel application types and network threats. This is also important in detecting subtle attacks, the increase in use of certain applications or shifts in application usage patterns, all of which are typically not visible for standard monitoring tools. While ML automates all these and helps unearth weak or inaccurate signatures, DL works through its many layers to automatically extract higher-level features with which network algorithms can be enhanced.

Future-proofing DPI to ensure encrypted traffic visibility

ipoque is already making great strides in both ML and DL by collaborating with top universities and its own distinguished team of data scientists who deploy advanced statistical and classical ML, high-dimensional data analysis and DL to future-proof our DPI software to enable encrypted traffic management and encrypted traffic visbility.

Operators looking to enhance their traffic management capabilities with ML and DL will benefit from incorporating our DPI software. The software can be embedded into any existing infrastructure, at any point in the network. R&S®PACE 2 enables operators to tap into the unlimited insights promised by the continuous advancements in the artificial intelligence space, In doing so, operators can drive the creation of a truly intelligent network capable of managing and sustaining itself in the future.

Sources

[1] https://www.fortinet.com/blog/industry-trends/keeping-up-with-performance-demands-of-encrypted-web-traffic