# Publishing outputs

Install Python 3.10+ and `pip install -r "$SKILL/scripts/requirements.txt"` in an
approved environment. Set `SKILL` to this skill's absolute directory. Supply an authorized `HF_TOKEN` through the site's secret mechanism; this can be
an owner-provided delegated token (no separate membership needed), or your own
token with organization access. Do not
paste it into a prompt or commit it. On SOCKS-proxy hosts add `httpx[socks]`.

Buckets are public by default: readers need no token or organization membership.
Write access still requires HF_TOKEN. Uploaded files are publicly downloadable.

## Files and checkpoints

Use a dedicated directory containing only the outputs intended for collaborators.
Select include patterns explicitly. The helper excludes common credential names,
but filenames cannot prove that a log or JSON is free of credentials.

```bash
python "$SKILL/scripts/share.py" folder --bucket "$BUCKET" \
  --folder "$OUTPUTS" --prefix "experiments/$EXPERIMENT/runs/$RUN_ID/artifacts" \
  --include '*.json' '*.log' '*.safetensors' --dry-run
```

Remove `--dry-run` for immediate upload; add `--watch --interval 30` for periodic
add/update synchronization. No destination deletion occurs. Files changing during
upload are not an atomic snapshot: write checkpoints into temporary directories,
finish all files, then atomically rename to `checkpoints/step-000100/` before
syncing. A trainer can invoke the one-shot command after every N steps with that
completed directory and a step-specific prefix. Never repeatedly overwrite one
checkpoint filename when earlier checkpoints must remain accessible.

Download with the installed HF CLI:

```bash
hf buckets sync "hf://buckets/$BUCKET/experiments/$EXPERIMENT/runs/$RUN_ID/artifacts" ./downloaded
```

Folder uploads alone do not create dashboard curves. Publish metrics separately.

## MLEvolve formal_mila adapter

`$ROOT/<dataset>/state.json` must contain `run_root`, `elapsed_seconds`,
`budget_seconds`, `updated_at`, and status; `$run_root/logs/journal.json` must contain
completed candidate nodes. Paths must be accessible from the uploader's host.
This is the formal_mila output contract, not all possible MLEvolve layouts.

```bash
python "$SKILL/scripts/mlevolve.py" mlevolve --bucket "$BUCKET" \
  --experiment "$EXPERIMENT" --root "$ROOT" --contributor "$CONTRIBUTOR" \
  --run-group "$RUN_GROUP" --method AQ05 --seed 42 --hardware A100 \
  --cache "$CACHE" --datasets dataset-a dataset-b --interval 30
```

Use `--once` for a one-shot upload. Keep `CACHE` and `RUN_GROUP` across resumes.
A distinct seed or independent repeat requires another run-group; otherwise IDs
collide. The adapter only reads experiment files, and never repairs training.
One process owns a contributor/run-group, protected by a local filesystem lock.
Use the same shared cache path if a restart runs on another node.

## Other training frameworks

Atomically write a per-run JSON snapshot with the schema below, then invoke:

```bash
python "$SKILL/scripts/mlevolve.py" publish --bucket "$BUCKET" \
  --experiment "$EXPERIMENT" --snapshot "$SNAPSHOT"
```

Call after completed candidate evaluations/checkpoints or run a project-owned
periodic producer. Do not blindly replay a snapshot and advance its source time.
The upload command updates only `uploaded_at`; the producer owns heartbeat time.
Store at `experiments/<experiment>/runs/<run_id>/live.json`. Each snapshot contains
all completed metric events for the run, and only one writer may update it.

```json
{
  "schema_version": 1, "experiment_id": "example-campaign",
  "run_id": "alice.repeat1.dataset-a", "dataset_id": "dataset-a",
  "contributor": "alice", "method_id": "AQ05", "seed": 42,
  "attempt_id": "scheduler-job-id", "hardware": "A100",
  "status": "running", "elapsed_seconds": 120, "budget_seconds": 43200,
  "source_updated_at": "2026-09-21T10:00:00+00:00",
  "uploaded_at": "2026-09-21T10:00:00+00:00",
  "metric_name": "validation accuracy", "maximize": true,
  "metrics": [{"event_id": "candidate-1", "step": 1, "value": 0.8}],
  "observations": [{"elapsed_seconds": 120, "value": 0.8}],
  "official_score": null, "error": ""
}
```

Examples are formats, never real results. Use timezone-aware source timestamps.
Statuses: starting, queued, running, paused, finalizing, completed, failed,
cancelled. Set `maximize` to true/false or null if unknown; do not guess the
metric direction. Event IDs are unique; points must be finite and step-sorted.
Keep validation and final grading separate. Unknown active runtime requires an
adapter/UI extension; do not invent elapsed or budget just to satisfy this schema.

API reference: [HF Bucket guide](https://huggingface.co/docs/huggingface_hub/guides/buckets).