CVE-2023-44487 HTTP/2 Rapid Reset Attack Investigation - Eventus

During August to October 2023, we investigated the vulnerability for clients. The purpose of the investigation was to explore a new method of exploiting the stream multiplexing feature in the HTTP/2 protocol. This method, known as "Rapid Reset," proved effective in volumetric DDoS attacks. This vulnerability, CVE-2023-44487, poses a highly severe threat with a CVSS score of 7.5.

Every web server using HTTP/2 is potentially at risk, and the flaw can lead to massive DDoS attacks. The web servers, CDN providers, and Cloud Providers have coordinated the efforts for mitigating this vulnerability.

What is CVE-2023-44487 HTTP/2 Rapid Reset Attack?

The 'Rapid Reset' technique utilizes a feature in HTTP/2 known as 'stream multiplexing' to overwhelm the server with numerous requests. The server promptly cancels these requests, resulting in a significant workload but minimal impact on the attacker. The attack overwhelms the target website or app by repeatedly sending and cancelling requests using a feature in HTTP/2. This causes the website or app to stop working properly.

HTTP/2 has a safety feature to reduce active streams and prevent DoS attacks, but it's not always effective. In this attack, the client exploits the protocol's allowance to cancel streams without needing the server's agreement. Botnets can generate massive request rates, posing a severe threat to targeted web infrastructures.

The CVE-2023-44487 vulnerability can overload web servers by quickly creating and cancelling streams, possibly causing a Denial of Service. Customers with HTTP/2-enabled web servers should contact their server vendors for required updates.

Background - RFCs

RFCs, or Request for Comments, are standardized documents produced by the Internet Engineering Task Force (IETF) that define various Internet-related protocols, guidelines, and procedures. There are related RFCs that are being misused about the vulnerabilities of HTTP/2 and its DDoS attack variants.

1. RFC 7540 - Hypertext Transfer Protocol Version 2 (HTTP/2)

• Description: This is the main RFC for HTTP/2, detailing its architecture, features, and protocols.
• Misuse:

1. Stream Mechanisms: HTTP/2 introduced the concept of multiplexing multiple streams over a single connection. Attackers exploit this by opening numerous streams concurrently to exhaust server resources.
2. Flow Control & Stream Priority: HTTP/2 offers sophisticated flow control and stream priority features. By mismanaging these or deliberately ignoring priority settings, attackers can further disrupt server operations.

2. RFC 7541 - HPACK: Header Compression for HTTP/2

• Description: This RFC details the header compression mechanism for HTTP/2, known as HPACK.
• Misuse:

1. Attackers can overload the HPACK decompression process by creating headers. This can cause the server to use more resources and potentially crash or become slower.

3. HTTP/2 RFC 9113

• Description: A syntax of HTTP that uses a binary framing format, which provides streams to support concurrent requests and responses. Message fields can be compressed using HPACK. Typically used over TCP and TLS. Obsoletes these two RFCs - 7540 and 8740.
• Misuse

1. Stream Limit Behaviour: RFC 9113 supposedly suggests that when a client tries to open more streams than allowed, only the excess streams should be invalidated and not the entire connection. This behaviour is exploited by attackers in the multiple streams opening attack. By continuously attempting to open more streams than the server's advertised limit, attackers force the server to keep managing and invalidating these streams while still maintaining an active connection (new requests repeatedly).

These RFCs were designed to enhance the performance and capabilities of HTTP/2, making web browsing faster, more efficient, and secure. However, as with many technologies, the features introduced can sometimes be misused or exploited for malicious purposes. In this case, attackers are taking advantage of the multiplexed nature of HTTP/2 and its specific behaviours, as defined in these RFCs, to launch sophisticated DDoS attacks.

Stream Concurrency Abuse

RFC 9113 discusses stream concurrency in HTTP/2 and the rules governing it. Specifically, it explains how to manage and enforce the maximum number of concurrently active streams. Let us explore the aspects of this segment that individuals can exploit or misuse, resulting in vulnerabilities.

• SETTINGS_MAX_CONCURRENT_STREAMS Parameter:

This parameter used by an endpoint sets a limit on the number of streams that a peer can start at the same time. Each endpoint specifies its own limit: the client specifies the limit for the server, and vice versa. This setting becomes an attack vector if not enforced properly. If an attacker ignores the limit and creates too many streams, it can overload the server or cause problems.

• Counting Toward Stream Limit:

Streams in the open state, or either of the half-closed states, count toward the maximum number of concurrently active streams. Those in the reserved states do not. An attacker can exploit this rule by keeping streams in a state that does not count toward the limit. This can potentially allow the server to handle more streams than it is capable of.

• Exceeding the Stream Limit:

Endpoints are strictly not allowed to exceed the concurrent stream limit set by their peer. If an endpoint receives a HEADERS frame that surpasses its concurrent stream limit, it should treat this as a stream error. Attackers can misuse this and intentionally send too many HEADERS frames to crash or slow down the system. Moreover, depending on the error code chosen

(PROTOCOL_ERROR or REFUSED_STREAM), the attacker might manipulate the server's behaviour, such as triggering automatic retries.

• Reducing the SETTINGS_MAX_CONCURRENT_STREAMS Value:

If an endpoint wants to decrease the SETTINGS_MAX_CONCURRENT_STREAMS value to a number below the current number of open streams, it has two options - either close the streams that exceed the new value or wait for streams to complete. An attacker can take advantage of this by opening many streams and making the server choose between closing some or waiting. Improperly implemented servers can experience induced indecision and errors, resource exhaustion, or even deadlocks and performance degradation when using this.

HTTP/2 Rapid Reset Attack Exploitation Technique (step by step) -

1. Initial Connection Establishment: The attacker connects to the server, getting ready to exchange data using the HTTP/2 protocol.
2. Stream Initiation & Rapid Cancellation: The malicious client then starts to exploit the RST_STREAM mechanism by initiating multiple HTTP/2 streams. But instead of maintaining these streams, the client rapidly sends an RST_STREAM frame for each, signaling the server to cancel the stream. By sending a massive number of requests followed by RST_STREAM frames (indicating a request cancellation), attackers can overwhelm servers. This creates a backlog of tasks for the server, consuming resources and leading to denial of service. The key here is the speed: streams are cancelled (reset) faster than new streams arrive.
3. Server’s Reaction and Vulnerability Exploitation: On the server's side, every incoming stream requires the allocation of certain resources, even if for a short period. Because the fast stream starts and stops, the server keeps allocating and then freeing resources in a loop. This rapid cycle causes undue stress on the server's system. Servers with a proxy or load balancer may be at risk. This is because the proxy can send requests before realizing they have been cancelled.
4. Bypassing Concurrency Limits: The concurrency limit cannot defend against this rapid reset method because the client can cycle through requests rapidly, irrespective of the server's concurrency settings. Servers often have a configured threshold for how many concurrent streams they can handle, denoted as http2_max_concurrent_streams. However, the attacker cleverly bypasses this limit.
   Because streams reset so quickly, the server's maximum threshold is never reached by the total number of concurrent streams. Thus, the overload misleads the server into a false sense of normality.
5. Final Impact: Over time, with the continuous barrage of initiated and cancelled streams, the server's resources, like CPU and memory, become strained. If this cycle persists and the server is not configured to identify and counteract such behaviour, it can lead to server degradation or a Denial-of-Service condition, where legitimate users are denied access to the server's services.

Types of Attack Variants in Rapid Reset Attack

Variant 1: Delayed Stream Cancellation

1. Attack Mechanism:

• The attacker opens too many streams all at once.
• Instead of cancelling them immediately (like in the regular Rapid Reset attack), there is a delay before cancelling these streams.
• Immediately after cancelling, another large batch of streams is opened, and the cycle continues.

2. Advantages & Efficiency:

• This variant tries to avoid detection by delaying the cancellation of streams, which can bypass certain protection methods. For instance, if a server has a rule to close a connection if it detects more than 100 RST_STREAM s per second, this delayed cancellation might sneak below that threshold.
• While this variant loses the advantage of maximum connection utilization, it retains some efficiency gains over the standard HTTP/2 DDoS attack because of the batch-based approach.

3. Mitigation Considerations:

• To effectively defend against this variant, any rate-limiting stream cancellation mitigation must set tight limits. The attacker wants to find a spot where they can avoid detection but still cause problems.

Variant 2: Multiple Stream Opening

1. Attack Mechanism:

• The attacker (client) disregards the server's advertised limit on the number of concurrent streams.
• The attacker (client) aggressively tries to open more streams than the server claims it can handle.

2. Advantages & Efficiency:

• This variant allows the attacker to keep the request pipeline saturated. By constantly pushing requests, the round-trip time (RTT) between the client and the proxy becomes inconsequential.
• If the server responds quickly to the requested resource, the round-trip time (RTT) between the proxy and server does not matter. This gives the attacker the ability to continuously pressure both the proxy and the server.

3. Implications and RFC 9113:

• RFC 9113, which outlines the HTTP/2 protocol guidelines, suggests that if a client tries to open more streams than allowed, only those extra streams should be invalidated, not the entire connection. This means that a compliant server will close only the streams that exceed its advertised limit while continuing to process other legitimate streams.
• This characteristic is what the attack exploits. The server wastes time handling unnecessary streams and stopping them. However, the connection is not completely ended. As a result, the attacker can continue sending new streams immediately after the previous ones are dealt with.

How can Eventus help?

Eventus Security team was involved in researching, understanding, and mitigating the CVE-2023-44487 HTTP/2 Rapid Reset attack for clients. This gives an upper hand for our clients to determine if they are vulnerable to such attack. As a recognized Managed Security Service Provider, and with a wealth of experience in next generation cyber security services, Eventus can help you with customized enterprise services & solutions for your business. Our team involves dedicated 24/7 of cyber experts who will work closely with you to ensure your security posture.