Paired PR. This is the open-source half of a two-repo change. The archiver switch lived in tuist/TuistCacheEE#69, now merged; this PR carries the cache-version bump, dependency wiring, EE storage tests, and the submodule pointer bump. The submodule points at the merged EE main commit 650dc80.
What
Compresses remote binary/module cache artifacts with AppleArchive/LZFSE instead of storing them in uncompressed zips, and bumps CacheVersion (v5 -> v6) so the format change is safe across CLI versions.
This PR:
- Bumps
CacheVersion.current to version6 in CacheVersionFetcher.
- Adds
TuistAppleArchiver to the TuistCacheEE target (and its test target) in Module.swift.
- Updates the
TuistCacheEETests storage tests to build AppleArchive fixtures instead of store-zip fixtures.
- Updates the
CacheVersionFetcher unit test to expect version6.
- Bumps the
cli/TuistCacheEE submodule pointer to the merged EE main commit 650dc80.
Why
A customer reported cold cache downloads around 7 GB. The cache artifacts are archived with ZIPFoundation’s default compressionMethod: .none, i.e. stored uncompressed - every zip entry is “Stored 0%”. The payloads are Mach-O binaries that compress very well, so downloads carried essentially the raw xcframework bytes. Root cause: FileArchiver -> FileSystem.zipFileOrDirectoryContent -> FileManager.zipItem(...) never passes a compressionMethod, and ZIPFoundation defaults it to .none. This was an unexamined default, not a deliberate decision.
Validated results
Benchmarked the three archivers on a real 2.45 GB local cache (340 xcframeworks), reproducing the exact upload/download call paths (store = today’s ZIPFoundation, deflate = zip with compression, LZFSE = AppleArchive):
| method |
size vs raw |
compress |
decompress |
| store (today) |
100% |
163 MB/s |
304 MB/s |
| deflate (zip) |
23.9% |
48 MB/s |
167 MB/s |
| LZFSE (this PR) |
22.2% |
277 MB/s |
445 MB/s |
- ~76% smaller archives than store on a representative cache (2.45 GB -> ~0.54 GB), which is the download saving.
- LZFSE compresses marginally smaller than deflate, ~6x faster to compress, and ~2.7x faster to decompress. Deflate was too slow to justify keeping the zip format (this matches the earlier decision to use
TuistAppleArchiver over deflate for sharding), and since the format change forces a cache-version bump regardless, LZFSE’s format-change cost is already paid.
- Every LZFSE archive round-tripped losslessly through
decompress.
Why a cache-version bump
AppleArchive (.aar) is a different, Apple-only format that older, zip-only CLIs cannot decompress. Without a version bump they would resolve a v5 key, download an .aar, and fail to unarchive it. Bumping to v6 moves the new artifacts into a key namespace old clients never resolve, so they cleanly take a cache miss and rebuild instead. Both the legacy and regional binary/module cache are covered because both derive item hashes from CacheGraphContentHasher, which folds the version into the hash. This implies a one-time mandatory cache re-warm on upgrade, which is expected and accepted.
Concurrency-safety fix (validated in EE #69)
Review of the archiver rewrite caught a regression: the upload batch is keyed by (hash, name) and can legitimately hold two items that share a hash with different names (package products), but the rewrite archived every item to temporaryDirectory/<hash>.aar in a shared batch directory. Run concurrently, those two items raced on the same path and could upload a truncated or wrong artifact. The old FileArchiver isolated each item in its own temp directory. Fixed by naming each archive after its per-item signature-stripped staging directory (a fresh UUID), restoring per-item isolation. Download paths were already safe (each uses its own per-call temp directory).
Impact
- ~76% smaller cold cache downloads for cache-heavy projects.
- One-time cache re-warm on upgrade to the CLI carrying v6 (expected and accepted).
- Lower egress on both the consumer and Tuist storage sides.
Validation
- Benchmarked store/deflate/LZFSE on the real 2.45 GB cache (numbers above); every LZFSE archive round-tripped losslessly.
TuistCacheEEUnitTests builds green against both storages after the concurrency fix (xcodebuild build-for-testing -scheme TuistCacheEEUnitTests -> TEST BUILD SUCCEEDED).
- Storage tests updated to build AppleArchive fixtures;
CacheVersionFetcher test updated to version6.
How to test locally
- Check out this branch with the submodule at the paired EE commit (
git submodule update --init cli/TuistCacheEE).
TUIST_EE=1 tuist generate Tuist TuistCacheEE --no-open, then build the Tuist-Workspace scheme.
- Run the EE storage suites:
TuistCacheEETests/CacheRemoteStorageTests and TuistCacheEETests/ModuleCacheRemoteStorageTests.
- End-to-end: warm a remote cache with this CLI, wipe the local cache, and confirm the pulled artifacts are
.aar, decompress correctly, and the download is materially smaller than on main.
Coordination checklist
- Merge tuist/TuistCacheEE#69.
- Re-point
cli/TuistCacheEE to the merged EE main commit (650dc80).
- If this branch is rebased onto a newer
main, keep the submodule at a commit whose EE tree matches the core it builds against.