Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

You can now run advanced AI models locally without needing expensive cloud infrastructure or specialized hardware.

The Strata project has released a one-click installer for Windows and Linux that allows users to run the Qwen 3.8 Flash Next (125B) large language model on consumer-grade hardware, such as an NVIDIA RTX 4090 or AMD RX 9070 XT graphics card. This typically server-bound model, which possesses 125 billion parameters, can now operate with a minimum of 12GB of VRAM and 32GB of RAM, making advanced AI accessible on personal computers. The Strata inference engine also provides an OpenAI-compatible API endpoint on localhost, enabling integration with various applications and coding agents. Performance benchmarks indicate that the system can achieve response speeds of up to 94 tokens per second for writing answers and 2,650 tokens per second for processing prompts on NVIDIA hardware, significantly faster than typical human reading speed. The software is free and open-source, with an optional donation system to support ongoing development.

The development of Strata signifies a major step towards democratizing access to powerful AI models, moving them from enterprise-level servers to personal workstations. This allows individuals and smaller development teams to experiment with and deploy large language models without incurring significant cloud computing costs. The project's success in enabling local execution of a 125-billion-parameter model suggests a broader trend of model optimization and efficient inference techniques that could benefit the entire AI community.

This accessibility shift directly impacts developers, researchers, and enthusiasts by removing the barriers of cost and complex setup associated with running high-parameter AI models. It enables faster iteration cycles for AI-powered applications and opens new possibilities for privacy-focused AI solutions, as data no longer needs to leave the user's machine. Future developments will likely focus on further optimizing model sizes and inference speeds to support even larger models on a wider range of consumer hardware.