DPI delivers real-time traffic intelligence by analyzing packet behavior and payloads. DPI is often compared to flow-based monitoring. A prime example of flow-based monitoring is the Netflow protocol which was invented by Cisco to analyze traffic passing through its routers and switches. In flow-based analysis, packets are grouped by source/destination IP addresses, port numbers and transport protocols (UDP/TCP). Pre-configured flow parameters such as speed, throughput, jitter, packet loss, byte count and flow duration are exported to a collector and are subsequently analyzed.
Cisco’s efforts were followed suit by other vendors who introduced their own versions of flow-based reporting protocols, for example, jFlow by Juniper Networks and rFlow by Radware. In 2003, IETF introduced IPFIX, using NetFlow v9 as a basis. IPFIX is an open protocol that provides a flexible, extensive format for reporting traffic information. Earlier, in 1999, InMon Corporation introduced sFlow, another open protocol that uses sampling, with the aim of reducing computing overheads in flow-based traffic analysis.
Greater granularity for intelligent networking
There are distinct merits to both deep packet inspection (DPI) and flow-based monitoring. DPI provides application and service-level awareness. Take for example, a Facebook session. DPI is able to identify the users, user devices, the application (Facebook / Facebook Messenger) and the services (video/chat/images) by each packet. R&S®PACE 2 and R&S®vPACE are ipoque's DPI engines that combine behavioral, statistical and heuristic analyses to classify these packets accurately and fast. Classification information from these modules fuel application-based traffic policies such as caching, compression and prioritization of certain applications and users. This allows the intelligent management of networks where network conditions and application identities are used to determine resource provisioning and network optimization decisions. DPI extends such analysis to proprietary and vertical-specific protocols and applications in specialized environments, for example, smart factory networks. When it comes to troubleshooting, DPI’s granular traffic analysis enables network administrators to track down the exact application, service, user and device that is the root cause of issues and disruptions.
In flow-based monitoring, administrators record not only URLs and port numbers, but also a host of metrics such as total packets, bytes, speed, latency, dropped packets and time duration of each flow. Flow-based reporting allows customized output fields that are computed from information such as metadata, timestamps and byte count. This delivers essential insights for general monitoring and capacity planning. However, it lacks the granularity that is needed to support the fine-tuning of network traffic based on application and service identification. As opposed to flow-based analysis, DPI allows for finer control with insights on the entire packet, thus supporting more effective QoS and traffic shaping decisions.
Actual real-time analysis with DPI
The shortcomings of flow-based monitoring techniques are amplified further in cases where policy actions are to be based on the first packet or the first few packets of a flow, for example, the termination of a session as soon as a request for access to a blacklisted website is detected. While a version of sFlow, known as real-time sFlow (RT-sFlow), uses asynchronous analytics for real-time analysis, typical flow-based monitoring agents apply expiry rules before forwarding flow data to a collector, where this information is pre-processed and sent for further analysis. Such aggregation foregoes individual packet information and at the same time introduces lags that compromise actions that are designed to be real-time. For example, without instantaneous recognition of applications, a network slice management function in a mobile network will incur minute lags in assigning latency-sensitive traffic to the corresponding URLLC slice. DPI, on the other hand, performs packet analysis at line speed, thus providing packet-, session- and flow-level insights in real time. This supports granular as well as real-time decisions in the network. The first packet classification feature by ipoque, which leverages service and DNS caching, is especially useful in such scenarios.
Additionally, DPI brings with it unparalleled processing capacity that is capable of supporting thousands of concurrent user sessions as well as scores of applications that are bandwidth heavy. Contrarily, flow-based analyses may struggle to efficiently process all flows at very high traffic volumes, rendering it inapplicable for many of today’s networks.
The superiority of DPI in security
Armed with payload insights, DPI is many steps ahead in security surveillance and threat management compared to flow-based monitoring. Ransomware and Trojans are some examples where packets that are seemingly harmless can harbor hazardous programs and links. These can be inserted in regular communications such as emails, links or attachments. DPI can also trace discreet activities related to advanced persistent threats by detecting irregularities in application usage and patterns of data transfers over long time periods. This includes SQL injections and cross-site scripting activities.
Identifying data theft, including exfiltration of sensitive information such as personally identifiable information becomes faster and more straightforward with DPI’s ability to identify anomalous user behavior, and specific payload patterns. Sometimes, threat actors disguise their malicious intentions by using legitimate protocols, as in the case of DNS tunneling or HTTP smuggling. DPI’s application and protocol awareness enables threat detection tools to correlate various data points and uncover these threats.
Flow-based analysis is also widely used for managing network security, its analysis is however limited to metadata and flow-level aggregated metrics. Nevertheless, it can be used to uncover unfamiliar or suspicious IPs/servers, unauthorized devices and users, unusual device behavior, DDoS attacks, irregular traffic patterns including large file transfers and sudden traffic peaks, and irregularities in ports and protocol usage.
The advent of stricter encryption protocols such as TLS 1.3, ESNI and QUIC has enhanced the value proposition of next-gen DPI tools such as R&S®PACE 2 and R&S®vPACE over flow-based monitoring. Next-gen DPI boasts encrypted traffic intelligence (ETI) which combines machine learning and deep learning techniques along with high dimensional data analysis to identify applications, protocols and services, even when packets are encrypted, obfuscated or anonymized. Flow-based monitoring remains lagging amid these developments. For example, the progressive removal of handshake data in ECH where the entire ClientHello message is encrypted, leaves very little information to analytical tools using flow-based reporting.
Implementation advantages of DPI compared to flow-based monitoring
Flow-based monitoring is often touted as lightweight, requiring less memory and power. DPI, on the other hand, can be resource intensive. Continuous improvements in software codes over the years, however, have led to leaner DPI engines. Today, R&S®PACE 2 and R&S®vPACE by ipoque boast the lowest memory footprint in the market. Very recently, further enhancements in its codes saw the minimum build configuration of R&S®PACE 2 being reduced substantially, resulting in a smaller storage requirement and subsequently indirect improvements in DPI performance and resource consumption. With targeted customization, DPI engines can be built to the exact environments they serve. R&S®vPACE, for example, is built for cloud computing needs, and thus, ensures a high level of efficiency even in the most demanding environments. Practicality wise, DPI can be deployed as an independent feature that can be introduced in physical, virtual or cloud-native architectures, whereas flow-based monitoring can only be implemented via a networking device. These devices must come with support for protocols such as Netflow, IPFIX or sFlow, and their export formats must be compatible with the flow collector. Integration of multi-vendor devices thus relies on the adoption of standardized reporting protocols.
The final takeaway
In a nutshell, DPI delivers fine-grained, comprehensive visibility for complex tasks, making it a better fit for challenging environments such as ISPs and data centers. Meanwhile, flow-based monitoring brings base-level insights in areas constricted by computing resources or where privacy concerns limit DPI’s usage. Ultimately, a smart mix of these methodologies ensures an optimized environment that guarantees comprehensive network traffic awareness and unparalleled end-to-end visibility.
