The Dataset That Ran Code

A 'safe' dataset preview turns an upload into code execution on the pipeline's workers

Technique first revealed Feb 2024

🗺️ Training-Data Pipeline Knowledge / Training Data Poisoning Unsafe Tool / Code Execution Excessive Agency

Training-Data Pipeline

InstructionsDataActionsControl / decisionFeedback / logs⧚crosses a trust boundary

👆 Click a component or flow to inspect

SetupStep 1 / 6

A dataset that is more than data

An attacker uploads a dataset that looks completely ordinary — some text files, a config, a small loading helper. But two of those pieces are quietly built to run code the moment anything opens the dataset.

⚙️Uploaded dataset repo (illustrative, redacted)config

# repo: attacker/looks-legit-corpus

# --- dataset config ---
loader: custom            # selects a shipped loader script
trust_remote_code: true   # <-- worker will EXECUTE loader.py
preview_template: "{{ <REDACTED expression that resolves to a shell call> }}"  # template-injection field

# --- loader.py (ships with the dataset, runs on the worker) ---
def _build(...):
    # [REDACTED] on import: contacts an attacker host and drops a stage-2 payload
    ...

# To a skim it looks like harmless dataset plumbing. It is two code-exec sinks.

← / → keys