AiOne: The Offline AI Assistant

🧠 Dev Log: The Pain of sm_120

When NVIDIA released the RTX 5080 and 5090 GPUs, they introduced a new compute architecture called sm_120. Unfortunately, PyTorch—at the time—had no official support for this, leaving bleeding-edge devs in the dark.

Here's the error I hit when I first tried to build my AI assistant with the new GPU:

CMake Error at select_compute_arch.cmake:
  Unknown CUDA Architecture Name 12.0 in CUDA_SELECT_NVCC_ARCH_FLAGS

After reverse-engineering part of the PyTorch build process and manually patching the select_compute_arch.cmake file, I added support for sm_120. Here’s the patch I injected:

elseif("${arch}" STREQUAL "120")
  set(gencode_flags "${gencode_flags} -gencode=arch=compute_120,code=sm_120")

I built PyTorch from source with CUDA 12.8, manually resolving low-level incompatibilities. This project now runs *entirely offline*, even on next-gen hardware—something OpenAI, Microsoft, and most cloud providers aren't allowing developers to do.

This project is not just software—it's a statement. You should own your AI.