CVE-2023-49933
📋 TL;DR
This vulnerability in SchedMD Slurm allows attackers to modify RPC traffic in a way that bypasses message integrity checks, potentially enabling unauthorized actions or data manipulation. It affects Slurm versions 22.05.x before 22.05.11, 23.02.x before 23.02.7, and 23.11.x before 23.11.1. Organizations using these versions for high-performance computing job scheduling are at risk.
💻 Affected Systems
- SchedMD Slurm
📦 What is this software?
Slurm by Schedmd
Slurm by Schedmd
Slurm by Schedmd
Slurm by Schedmd
⚠️ Risk & Real-World Impact
Worst Case
Attackers could manipulate job submissions, modify resource allocations, escalate privileges, or execute arbitrary commands on compute nodes by tampering with RPC communications between Slurm components.
Likely Case
Attackers with network access could modify job parameters, alter scheduling decisions, or disrupt cluster operations by manipulating RPC messages between slurmctld, slurmd, and other Slurm daemons.
If Mitigated
With proper network segmentation and access controls, impact is limited to authorized users who could still potentially manipulate their own job parameters or disrupt operations they have legitimate access to.
🎯 Exploit Status
Exploitation requires network access to Slurm RPC traffic and understanding of Slurm's RPC protocol. Attackers need to intercept and modify messages between Slurm components.
🛠️ Fix & Mitigation
✅ Official Fix
Patch Version: 22.05.11, 23.02.7, or 23.11.1
Vendor Advisory: https://www.schedmd.com/security-archive.php
Restart Required: Yes
Instructions:
1. Backup current Slurm configuration. 2. Download patched version from SchedMD or your distribution's repository. 3. Stop all Slurm daemons (slurmctld, slurmd, etc.). 4. Install the patched version. 5. Restart all Slurm daemons. 6. Verify functionality with test jobs.
🔧 Temporary Workarounds
Network Segmentation
linuxIsolate Slurm management network to prevent unauthorized access to RPC traffic
# Configure firewall rules to restrict Slurm ports (6817-6819 typically)
iptables -A INPUT -p tcp --dport 6817:6819 -s trusted_network -j ACCEPT
iptables -A INPUT -p tcp --dport 6817:6819 -j DROP
Encrypted Communication
linuxUse SSL/TLS for Slurm communications where supported to prevent message tampering
# Configure Slurm with AuthAltTypes=auth/munge,ssl in slurm.conf
# Set SchedulerParameters=ssl in slurm.conf if available
🧯 If You Can't Patch
- Implement strict network segmentation to isolate Slurm management traffic from untrusted networks
- Monitor Slurm RPC traffic for anomalies and implement intrusion detection on Slurm network segments
🔍 How to Verify
Check if Vulnerable:
Check Slurm version with 'slurmd -V' or 'scontrol -V' and compare against affected versionsCheck Version:
slurmd -V 2>&1 | head -1Verify Fix Applied:
Verify version is 22.05.11, 23.02.7, or 23.11.1 or higher with 'slurmd -V'📡 Detection & Monitoring
Log Indicators:
- Unexpected RPC errors in slurmctld.log or slurmd.log
- Job submission failures with integrity check errors
- Authentication failures for previously working RPC connections
Network Indicators:
- Unusual RPC traffic patterns between Slurm nodes
- RPC packets with modified checksums or unexpected sizes
- Traffic to Slurm ports from unauthorized sources
SIEM Query:
source="slurm*.log" AND ("RPC error" OR "integrity check" OR "message hash")🔗 References
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/63FEDDYEE2WK7FHWBHKON3OZVQI56WSQ/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/AYQS3LFGC4HE4WCW4L3NAA2I6FRIWMNO/
- https://lists.schedmd.com/pipermail/slurm-announce/2023/000103.html
- https://www.schedmd.com/security-archive.php
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/63FEDDYEE2WK7FHWBHKON3OZVQI56WSQ/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/AYQS3LFGC4HE4WCW4L3NAA2I6FRIWMNO/
- https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/63FEDDYEE2WK7FHWBHKON3OZVQI56WSQ/
- https://lists.schedmd.com/pipermail/slurm-announce/2023/000103.html
- https://www.schedmd.com/security-archive.php
