Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU

Large language models can now run on your existing 64GB RAM computer without needing a discrete GPU.

A new open-source project, llama-modchip, enables running large language models (LLMs) like the 284 billion parameter DeepSeek-V4-Flash and 30 billion parameter Mixture of Experts (MoE) models on consumer-grade hardware with only 64GB of RAM and no GPU. Developed by synchopate, the project utilizes advanced quantization techniques, such as Fractal-64 (3.75 bpw) and MXFP4 (4.25 bpw), along with CPU-specific optimizations including SwiGLU-H2 compression and Stockfish-PVS L3. This allows for large context windows of up to 256K tokens to be processed entirely in RAM, avoiding slower NVMe swap. The project also introduces a zero-API Claude Code restoration feature that significantly reduces prompt re-evaluation latency, bringing it down to milliseconds. The llama-modchip project provides pre-compiled binaries and a one-command setup for Linux and macOS, with a hybrid GPU+CPU pack for Windows.

The primary breakthrough of llama-modchip lies in its aggressive optimization and quantization strategies, enabling models with hundreds of billions of parameters to fit within the memory constraints of typical desktop or server hardware. By employing techniques like Analytical SwiGLU-H2 Second-Order Hessian Compression and custom MLA transcoding, the project dramatically reduces the memory footprint of models like DeepSeek-V4-Flash from over 76GB to under 60GB. This innovation democratizes access to powerful LLMs, shifting the landscape from GPU-bound accessibility to CPU-centric deployment for a wider range of users and applications.

This development has significant implications for local AI development and deployment, particularly for users who cannot afford or access high-end GPU hardware. The ability to run large models on readily available CPUs with substantial RAM not only lowers the barrier to entry but also opens up possibilities for real-time inference and complex agentic workflows without relying on cloud services. Future advancements may focus on further optimizing these models for even lower memory consumption or increasing inference speeds on diverse CPU architectures.