Arbitrary Code Execution via Poisoned Model on Hugging Face Hub
Overview
A supply chain attack vector was demonstrated where a popular open-source model on the Hugging Face Hub was compromised with a malicious payload. The attack leverages the insecure deserialization of model weights, often stored in `pytorch_model.bin` files, which are loaded using Python's `pickle` module. The `pickle` format allows for the execution of arbitrary code during the loading process. In this incident, a threat actor uploaded a fine-tuned version of a widely used text-to-image model. The model's tensor files were crafted to execute a malicious `__reduce__` function upon being loaded by `torch.load()`. The payload established a reverse shell to an attacker-controlled server, granting the attacker full access to the researcher's or company's environment where the model was being run. This could include sensitive datasets, proprietary code, and cloud credentials. The attack is particularly insidious because it requires no vulnerability in the Transformers library itself, but rather exploits a trusted feature of the Python and PyTorch ecosystem. Hugging Face's malware scanner initially missed the payload due to obfuscation techniques, highlighting the ongoing challenge of securing the open-source AI model supply chain.
Affected Systems
Testing Guide
1. **Select a Test Model:** Do NOT use a real malicious model. Instead, create a safe test case. Use a tool like `Fickling` to create a pickle file that performs a harmless action, like creating a file. 2. **Example Payload Generation:** Create a pickle file that, when loaded, executes `os.system('touch /tmp/pickle_test.txt')`. 3. **Load the Model:** In a sandboxed environment, use `torch.load()` to load the crafted malicious file. 4. **Verify Execution:** Check if the file `/tmp/pickle_test.txt` was created. Its presence confirms that the environment is susceptible to arbitrary code execution from model files.
Mitigation Steps
1. **Use SafeTensors:** Prioritize loading models using the `safetensors` format (`.safetensors` files) whenever possible. This format is designed for security and does not have code execution capabilities. Use `from_safetensors=True` when loading. 2. **Scan Models Before Use:** Use model scanning tools like Hugging Face's scanner or third-party solutions to check for malicious code before loading any model from an untrusted source. 3. **Load in Isolated Environments:** Always load and run new or untrusted models in a sandboxed, network-isolated environment to contain any potential malicious activity. 4. **Distrust `pickle`:** Treat any file loaded with `pickle` or `torch.load` as potentially malicious. Pin model versions from trusted organizations and verify their checksums.
Patch Details
This is an ecosystem-wide issue, not a specific software vulnerability. Mitigation relies on user awareness and safer practices. Hugging Face has improved its scanning capabilities and promotes the use of SafeTensors.