Remote Code Execution via Malicious Pickle Payloads in Hugging Face Hub Models
Overview
A recurring critical security risk within the AI ecosystem was highlighted by the widespread use of Python's `pickle` format for model serialization on platforms like Hugging Face Hub. The `pickle` module is known to be insecure and can execute arbitrary code during deserialization (unpickling). An attacker can create a malicious model file (e.g., `pytorch_model.bin`) containing a pickled object that, when loaded by a victim using standard library functions like `torch.load()`, executes an embedded payload. This payload can establish a reverse shell, steal credentials and SSH keys from the user's environment, or install persistent malware. The attack vector is particularly potent because downloading and loading pre-trained models is a standard practice for ML engineers and researchers. A user might innocently download a seemingly useful model from the Hub, only to have their machine compromised upon loading it. While Hugging Face has implemented security scanners to detect malicious pickles and heavily promotes the use of the secure `safetensors` format, legacy models and the inherent trust users place in the platform remain a significant risk. The incident served as a major catalyst for the community to migrate away from `pickle` for sharing model weights, emphasizing the need for secure-by-default practices in the MLOps lifecycle.
Affected Systems
Testing Guide
1. **Review Codebase:** Search your code for instances of `torch.load(`, `pickle.load(`, or `tf.keras.models.load_model(`. 2. **Check Loading Parameters:** For `torch.load`, verify if the `weights_only=True` argument is used. If it is not, the loading process is potentially unsafe. 3. **Analyze Model Sources:** Identify all models that your applications download from public hubs. Check if you are downloading them in `pickle` format (`.bin`, `.pkl`) instead of `.safetensors`. 4. **Simulate with a Safe Payload:** (For advanced users in a secure environment) Create a harmless pickle file that simply prints a message upon loading (`cposix system (S'echo "Pickle RCE is possible"' tR.`) and attempt to load it with your application's logic to see if the command executes.
Mitigation Steps
1. **Use `safetensors`:** Prioritize loading models exclusively in the `.safetensors` format. When loading PyTorch models, use `torch.load('model.bin', weights_only=True)` if a safetensors version is unavailable to limit deserialization to just the tensor data. 2. **Scan Models Before Use:** Use security tools like `picklescan` to scan model files for dangerous opcodes before loading them. 3. **Isolate Loading Environments:** Load new, untrusted models in a sandboxed, network-isolated environment (e.g., a temporary Docker container) to prevent a potential compromise from affecting the host system or internal network. 4. **Verify Model Source:** Only use models from official, verified creators on Hugging Face Hub and be cautious of newly uploaded models or those from unknown authors.
Patch Details
This is a design-level risk in the Python ecosystem. Mitigation relies on adopting safer formats like `safetensors` and secure coding practices.