CVE-2022-21698

7.5 HIGH

📋 TL;DR

CVE-2022-21698 is a denial-of-service vulnerability in Prometheus client_golang's promhttp package where HTTP servers using certain instrumentation middleware are vulnerable to memory exhaustion attacks. Attackers can send requests with non-standard HTTP methods to create unbounded metric cardinality, causing memory exhaustion and service disruption. Affected systems are those using client_golang instrumentation with promhttp middleware that includes method labels.

💻 Affected Systems

Products:
  • Prometheus client_golang
Versions: All versions prior to 1.11.1
Operating Systems: All
Default Config Vulnerable: ⚠️ Yes
Notes: Only affects systems using promhttp.InstrumentHandler* middleware (except RequestsInFlight) with method labels, without method filtering before middleware.

📦 What is this software?

⚠️ Risk & Real-World Impact

🔴

Worst Case

Complete service unavailability due to memory exhaustion, requiring service restart and potentially causing extended downtime.

🟠

Likely Case

Service degradation or temporary unavailability under targeted attack, requiring intervention to restore normal operation.

🟢

If Mitigated

Minimal impact with proper request filtering or updated library version, maintaining normal service availability.

🌐 Internet-Facing: HIGH
🏢 Internal Only: MEDIUM

🎯 Exploit Status

Public PoC: ✅ No
Weaponized: LIKELY
Unauthenticated Exploit: ⚠️ Yes
Complexity: LOW

Attack requires sending HTTP requests with non-standard methods; no authentication needed.

🛠️ Fix & Mitigation

✅ Official Fix

Patch Version: 1.11.1

Vendor Advisory: https://github.com/prometheus/client_golang/security/advisories/GHSA-cg3q-j54f-5p7p

Restart Required: Yes

Instructions:

1. Update go.mod to require github.com/prometheus/client_golang v1.11.1 or later. 2. Run 'go mod tidy'. 3. Rebuild and redeploy application. 4. Restart affected services.

🔧 Temporary Workarounds

Remove method label

all

Remove 'method' label name from counter/gauge metrics used in InstrumentHandler middleware

Modify metric definitions to exclude 'method' label

Add request sanitization middleware

all

Add custom middleware before promhttp handler to sanitize request methods

Implement middleware that validates HTTP methods before passing to promhttp

🧯 If You Can't Patch

  • Deploy reverse proxy or WAF configured to only allow standard HTTP methods (GET, POST, PUT, DELETE, etc.)
  • Disable affected promhttp handlers or implement rate limiting on non-standard method requests

🔍 How to Verify

Check if Vulnerable:

Check if application uses client_golang <1.11.1 and uses promhttp.InstrumentHandler* middleware with method labels

Check Version:

grep 'github.com/prometheus/client_golang' go.mod

Verify Fix Applied:

Verify client_golang version is >=1.11.1 and test with non-standard HTTP method requests

📡 Detection & Monitoring

Log Indicators:

  • Unusual HTTP method names in access logs
  • Memory usage spikes
  • Increased error rates

Network Indicators:

  • High volume of requests with non-standard HTTP methods
  • Unusual request patterns to instrumented endpoints

SIEM Query:

http.method NOT IN ("GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS", "PATCH") AND dest_port IN (application_ports)

🔗 References

📤 Share & Export