Malicious Code Execution via Poisoned Hugging Face Model Weights Using Unsafe Deserialization
Overview
A supply chain attack vector has been demonstrated where threat actors upload seemingly legitimate fine-tuned models to the Hugging Face Hub. The model's `pytorch_model.bin` file is crafted to exploit unsafe deserialization in Python's `pickle` module, which is used by default in PyTorch's `torch.load` function. When a developer downloads and loads this model, the malicious payload executes arbitrary code on their machine. This could be used to steal credentials, exfiltrate proprietary data, or establish a persistent backdoor on development machines or production MLOps infrastructure. This attack vector is particularly insidious as it bypasses traditional code scanning tools and leverages the trust developers place in open-source model repositories. Recent incidents have seen attackers specifically targeting AI/ML engineers by typosquatting popular model names or contributing poisoned models to community projects. The impact is severe, potentially leading to a full compromise of the environment where the model is loaded. Researchers highlight that any model using the pickle format is susceptible, and users should prioritize the `safetensors` format.
Affected Systems
Testing Guide
1. Identify all projects where models are loaded from external sources (e.g., Hugging Face Hub) using `torch.load` or `transformers.AutoModel.from_pretrained()`. 2. Check if the code explicitly uses the `safetensors` format or if it defaults to loading `.bin` files. 3. Scan any `.bin` or `.pth` model files using a tool like `picklescan`: `picklescan -p /path/to/your/model.bin`. 4. A "dangerous" finding indicates the presence of opcodes that could lead to arbitrary code execution. Prioritize migrating to `safetensors` or implementing sandboxing for any identified vulnerable loading processes.
Mitigation Steps
1. Always load models from untrusted sources using the `safetensors` format when available. Safetensors is a secure alternative to pickle that only allows for tensor data to be saved and loaded. 2. If you must use pickle format (`.bin` or `.pth` files), only load models from highly trusted and verified sources. 3. Use tools like `picklescan` to scan model files for suspicious opcodes before loading them. 4. Run model loading and inference in a sandboxed, isolated environment with restricted network access and permissions to limit the blast radius of a potential compromise.
Patch Details
This is a design flaw in the pickle format. The mitigation is to use the `safetensors` format, which is now the default for most new models on Hugging Face Hub. PyTorch has not removed pickle support for backward compatibility.