TLS 1.3, ESNI, ECH and QUIC: Taming new age cryptography with DPI and AI/ML-based encrypted traffic intelligence

Man-in-the-middle attacks take place when a data transmission is intercepted by malicious third parties. Such eavesdropping activities pose a huge security threat to enterprises and end users because confidential and sensitive information can be leaked or held ransom, leading to severe reputation or financial losses.

Wrapping up IP packets in complex algorithms or routing them via a sealed passage helps network administrators and application owners safeguard their data. This practice led to the emergence of various traffic concealing techniques, such as obfuscation and anonymization, which helps keep the underlying information away from prying eyes while packets navigate public domains and unprotected routes. Encryption, in particular, has become a favorite across networks, where cryptographic ciphers encode data into unreadable cipher text.

DPI and encryption: friends or foes?

Encryption, by its very nature, makes traffic analysis a challenging task. It obscures parts of the information that used to be accessible to network monitoring tools such as deep packet inspection (DPI) software.

Prior to encryption, DPI software simply tapped into header information and matched traffic patterns to identify the underlying protocols and applications. Advanced DPI engines, such as R&S®PACE 2 and R&S®vPACE by ipoque, combine these techniques with behavioral and statistical/heuristic analysis to deliver comprehensive insights into traffic flows, including the detection of malicious, anomalous and suspicious flows. These methods had enabled ipoque to tackle early encryption protocols without much challenge. Behavioral data, such as packet sizes, packet rates and interpacket delay, combined with statistical/heuristic information, including the mean, median and variation of behavioral data and flow entropy, revealed the underlying applications despite encryption.

New protocols on the rise

In more recent years, however, encryption protocols started becoming more aggressive, resulting in progressively more information layers being concealed. Transport Layer Security (TLS) 1.3, which is predicted to account for the majority of web applications, removes weak cipher suites and only supports algorithms with perfect forward secrecy. This, coupled with its shortened handshake process and its option for 0-RTT session resumption, results in traffic becoming increasingly indiscernible. Encrypted Server Name Indication (ESNI), an add-on to TLS 1.3, encrypts SNI information in the ClientHello portion of the handshake, effectively hiding the intended server hostname.

Encrypted ClientHello (ECH) goes a step further and encrypts the entire ClientHello, obscuring most of the connection details from any packet monitoring tools. There is also a growing adoption of Quick UDP Internet Connections (QUIC). Introduced in 2012, QUIC runs TLS over UDP and is expected to overtake TCP usage on the web in the next decade. QUIC encrypts the transport layer and boasts a significantly shorter handshake process with built-in encryption from the first connection. The protocol thus eliminates most data from the network radar, leaving conventional DPI tools clueless of the applications and services navigating the network.

The guessing game gets serious

Google’s Transparency Report¹ reveals that 96 % of webpages accessed on a Chrome browser over Mac are encrypted – eight years ago, this figure was only 43 %. According to the same report, only 3 of the 100 most frequently visited websites still allow unencrypted browsing.

This may sound ironic, but the higher the share of encrypted traffic and the more stringent the encryption protocols, the more vulnerable networks become to cyber threats. These dangers arise from two situations: malware and ransomware can be smuggled freely via encrypted packets; and legitimate encrypted applications can be manipulated to execute attacks on the network.

Our recent report on encryption found that more than 85 % of networking vendors are grappling with visibility issues caused by new protocols, such as TLS 1.3, TLS 1.3 0-RTT, DNS-over-X (DoX) and ESNI. A wide range of network outcomes, especially security-related ones, are therefore compromised. The study also found decryption via SSL/TLS inspection falling short in addressing encrypted traffic visibility due to confidentiality, security, privacy regulations and performance concerns.

DPI proves its mettle

Despite the visibility issues networking vendors already face, we expect new forms of cyber threats to continuously push for tougher encryption protocols. At ipoque, we see this as an opportunity to further perfect our DPI technology. We leverage advanced AI techniques modeled upon the correlation of unencrypted data points, such as packet headers, traffic volume, flow data and unencrypted handshake data as well as packet-level and network-level metadata. This effort culminated in our encrypted traffic intelligence (ETI), a breakthrough method that combines statistical, behavioral and heuristic analysis with machine learning (ML), deep learning (DL), high-dimensional data analysis and advanced caching. ML algorithms include k-nearest neighbors and decision tree models, while DL algorithms cover convolutional neural networks, long short-term memory and recurrent neural networks.

ETI enables seamless identification of encrypted traffic, down to protocols, applications and services. It also tackles traffic that is obfuscated, for example through tunneling and domain fronting, or anonymized by VPNs, CDNs or proxies. ETI is available in both our DPI engines, allowing existing and future customers to incorporate encrypted traffic analysis and visibility in any part of the network.

Where ETI makes all the difference for DPI

This brings us to the next question. How will ETI influence the demand for deep packet inspection? Based on our research findings, there is a strong demand for next-gen DPI with encrypted traffic visibility among networking and cybersecurity vendors, with about 92 % of them acknowledging this requirement. We therefore see ETI as a major move in strengthening DPI’s role as a monitoring tool across today’s networks, specifically for real-time application awareness.

Take the case of SSE for example, where ETI complements solutions such as SWG, CASB and ZTNA. The identification of encrypted enterprise applications enables ZTNA policies to be executed according to application-based access rules. For CASBs, access to cloud-based encrypted applications becomes seamless as they are identified in real time and matched against the allowlist. Similarly, a SWG can tap into the insights provided by next-gen DPI software to manage requests for encrypted on-premises applications.

ETI also plays an important role for telecom networks. 5G network slicing requires application awareness for packets to be routed through virtual network instances. Within the 5G URLLC slice, for example, applications such as remote surgery, autonomous driving, augmented reality and mobile gaming are typically encrypted with the following protocols: Secure Real-time Transport Protocol, Datagram Transport Layer Security and Constrained Application Protocol. Research by the Basque Research & Technology Alliance and the University of the Basque Country² lists DPI as the key solution for traffic classification in network slicing, being able to identify protocols such as Real Time Protocol for video, Message Queue Telemetry Transport for IoT and Hypertext Transfer Protocol for web data. The research highlights encrypted traffic visibility as a pre-requisite for any DPI and AI/ML-based network intelligence solution deployed in 5G.

Hello quantum computing

As network intelligence solutions continue to evolve, so does the field of cryptology. Quantum computing, for example, will push for harder to break post quantum cryptography so that encrypted traffic remains resistant to cryptanalytic attacks.³ These will result in a new generation of encryption protocols, necessitating a DPI and AI/ML-based technology that can evolve faster and one that is always a step ahead in the encryption race.

Check out our whitepaper ‘Encrypted traffic visibility’ to stay ahead of the game!

Sources

[1] Google Transparency Report - https://transparencyreport.goo...
[2] Traffic Classification for Network Slicing in Mobile Networks - https://www.mdpi.com/2079-9292...
[3] Post-quantum cryptography - https://en.wikipedia.org/wiki/...