PrismML releases Bonsai 2 27B, which compresses Alibaba's Qwen3.8 27B to 5.9 GB, small enough for smartphones, while retaining 98.2% of Qwen's benchmark scores

Large language models can now run on personal devices without sacrificing significant performance.

PrismML has released Bonsai 2 27B, a compressed version of Alibaba's Qwen3.8 27B large language model. The new model is only 5.9 GB, a 9x to 10x reduction in size, making it small enough to potentially run on smartphones. Bonsai 2 retains 98.2% of Qwen's benchmark performance scores, a slight improvement from PrismML's previous model which achieved 95% parity. PrismML's technology, developed by Caltech researchers, achieves this by using a "ternary" weight system that simplifies model weights to three values (+1, -1, or 0) instead of the usual 16 bits. The company, advised by Databricks co-founder Ion Stoica and backed by investors like Khosla Ventures, aims to compress even larger models in the future.

PrismML's breakthrough in LLM compression, specifically Bonsai 2 27B, signals a significant shift towards making powerful AI accessible on edge devices. By reducing model size to under 6 GB while maintaining nearly all performance, PrismML challenges the prevailing notion that advanced AI requires massive cloud infrastructure. This development could democratize access to sophisticated AI capabilities, enabling applications that are both cost-effective and privacy-preserving for end-users.

The success of PrismML's "ternary" weight compression technique suggests a potential new standard for model efficiency, impacting hardware design and software deployment strategies. Companies relying on cloud-based LLMs may face increased competition from on-device solutions, necessitating a re-evaluation of their operational costs and data handling practices. The next phase to watch will be PrismML's application of this technology to hundreds of billions of parameters, potentially unlocking unprecedented AI capabilities on consumer electronics.