A script that skips the 20-minute llama.cpp compile on 4-core Colab sessions.
I burned a lot of Colab sessions recompiling llama.cpp every time: 4 cores, 20+ minutes each. So I scripted the painful part away. Now a fresh cloud instance is usable in minutes.
The official llama.cpp releases ship pre-built Windows CUDA binaries and nothing for Linux. Run llama.cpp across cloud instances and you recompile from source on every machine: 4 cores, 20+ minutes each, every single time. That is how I burned most of my early Colab sessions.
Build once per GPU SM version, store the binary on GitHub Releases, pull it anywhere in seconds. pull.sh detects the GPU, finds the matching binary, verifies its SHA256 checksum, and installs it to ~/.local/bin/llama. The archive bundles the CUDA runtime, so the target machine only needs a compatible driver.
Twenty minutes of compile time, every session, is a tax you only notice once you stop paying it.
Six scripts cover the whole lifecycle: detect.sh for diagnostics, build.sh for compiling and uploading, pull.sh for installing, list.sh for browsing the store, verify.sh for checksums, and cleanup.sh for removing old versions. configs/gpu_map.json maps GPU model names to SM versions, and CI auto-builds every SM version on each new llama.cpp release.
T4, A100, L40S, RTX 4090, H100: if it has an SM number, it has a binary. The repo has picked up a small following, and the pattern generalizes to any project that ships Windows-only binaries.