hf-xet 1.8.0 starts uploading xorbs straight to S3, and your firewall may need a new rule

Hugging Face shipped hf-xet 1.8.0 on 30 October, with wheels on PyPI from about 10:04 UTC. hf-xet is the Rust client that huggingface_hub uses to move large files to and from Xet storage on the Hub. The headline change is in uploads, and the release calls it experimental.

What changes on upload

Xet splits files into chunks and packs them into “xorbs.” Until now the client sent each xorb to Hugging Face’s CAS service. In 1.8.0 the client can instead ask CAS for a short-lived upload grant and then write the xorb directly to S3 at the address the grant names. CAS still validates each xorb before it counts. Per the release, the new path is switched on gradually from the server side, so two machines on the same version can behave differently for a while.

The groundwork is in PR #1004. The client posts the xorb length and a CRC-64/NVME checksum to a grant endpoint. A 201 means a grant, a 200 means the xorb already exists, and a 404 means the grant API is not available, in which case it falls back to the old route. The upload to the grant URL goes out without your Hugging Face token attached, and presigned query strings are now stripped from logs and error messages.

What to do

• Locked-down egress: allow s3.us-east-1.amazonaws.com, plus s3.eu-west-3.amazonaws.com if you push to EU-hosted repos. The release says uploads fall back to the previous path if the direct route fails, but a blocked host can still cost you retries and time.
• Want the old behaviour: set HF_XET_CLIENT_LEGACY_DIRECT_XORB_UPLOAD=1.
• Downloads are not part of this change.

The release also fixes two edge cases. The client no longer asks for reconstruction past the end of a file (issue #995), and it no longer publishes a partially written file when a download fails (issue #1067). It extends the transfer telemetry that arrived in hf-xet 1.7.0 with new counters, so check that post if you already set the telemetry opt-out. Current huggingface_hub releases accept any hf-xet from 1.6.0 up to 2.0, so a fresh install or pip install -U picks up 1.8.0 without a hub upgrade.

Why this matters

Most teams will never notice, because the Hub decides when a repo moves to the direct path and the client falls back on failure. The teams that will notice are the ones pushing checkpoints from CI or a training cluster behind an allowlist, where the first sign could be slow or retried uploads. Adding two S3 hostnames, or pinning the legacy path with one environment variable, is cheaper than debugging that mid-run. hf-xet is one of several Rust components under Hugging Face’s Python tools, alongside tokenizers, so its release notes are worth a skim with every bump.