MLflow Path Traversal Vulnerability Allows Arbitrary File Write and RCE
Overview
A high-severity path traversal vulnerability was found in the MLflow Tracking Server. The vulnerability, identified as CVE-2023-6958, exists in the `log_artifact` and `log_artifacts` API endpoints. Authenticated users with permission to log artifacts to a run can craft a malicious `artifact_path` containing `../` sequences. The server fails to properly sanitize this path, allowing the user to traverse outside of the intended artifact storage directory. This flaw enables an attacker to write arbitrary files to any location on the server's filesystem where the MLflow server process has write permissions. The impact is severe, as an attacker could overwrite critical system files. For example, they could overwrite a user's `.bashrc` or `.ssh/authorized_keys` to gain persistent access and execute arbitrary code upon the next user login. Alternatively, they could plant a web shell in a web server's document root if one is present on the same machine. This vulnerability underscores the importance of rigorous input validation in MLOps platforms, as they often handle file operations and are a prime target for attackers seeking to compromise AI development infrastructure.
Affected Systems
Testing Guide
1. **Check MLflow Version**: Run `mlflow --version` to see if your installed version is below 2.8.1. 2. **Set up a Test Run**: In a controlled environment with a vulnerable MLflow server, create a new experiment and run. 3. **Craft a Malicious Request**: Use the MLflow client to log an artifact with a crafted path. Example Python code: `import mlflow mlflow.set_tracking_uri("http://your-mlflow-server:5000") with open("payload.txt", "w") as f: f.write("pwned") with mlflow.start_run() as run: mlflow.log_artifact("payload.txt", artifact_path="../../../../../../tmp/pwned.txt")` 4. **Verify File Creation**: Check if the file `pwned.txt` was created in the `/tmp/` directory on the server. If it was, the server is vulnerable.
Mitigation Steps
1. **Upgrade MLflow**: Immediately upgrade your MLflow instance to version `2.8.1` or newer, which contains the patch for this vulnerability. 2. **Run as Low-Privilege User**: Run the MLflow server process as a dedicated, low-privilege user with restricted write permissions across the filesystem. 3. **Network Segmentation**: Isolate the MLflow Tracking Server from direct exposure to the internet. Place it behind a reverse proxy or VPN and use a firewall to restrict access to trusted IP ranges. 4. **Regularly Audit Permissions**: Periodically review the file permissions on the server hosting MLflow to ensure they adhere to the principle of least privilege.
Patch Details
Patched in MLflow version 2.8.1 by adding proper path validation and sanitization.