PrismML hopes its tiny LLM will change how we all use AI

AI models now fit on your personal computer, making advanced capabilities available offline and private.

PrismML, a startup founded by Caltech researchers, has released Bonsai 2 27B, a compressed version of Alibaba's Qwen3.8 27B model. This new model reduces the memory footprint by 9x to 10x, fitting into a 5.9 GB file size, making it suitable for PCs and potentially high-end smartphones. The company claims its proprietary compression technique, which simplifies model weights to ternary values (+1, -1, or 0), results in minimal performance loss, matching 98% of the original model's benchmark scores. PrismML's previous models have seen significant download numbers, indicating market interest in smaller, capable LLMs. CEO Babak Hassibi stated their next goal is to apply this compression to even larger, several-hundred-billion-parameter models, expecting easier performance retention with increased model size. The company is reportedly in discussions with Apple, though this remains unconfirmed.

PrismML's innovation signifies a crucial shift towards democratizing access to powerful AI by enabling on-device processing. This directly challenges the cloud-centric model, reducing reliance on remote servers and potentially lowering operational costs for users. The ability to run sophisticated LLMs locally promises enhanced privacy and offline functionality, appealing to a broad consumer base and enterprises prioritizing data security. The startup's focus on achieving near-perfect performance parity with much larger models suggests a new frontier in efficient AI deployment.

The success of Bonsai 2 27B, particularly its high benchmark scores and substantial memory reduction, indicates a growing market demand for smaller, more accessible AI models. This development could spur further research and investment into LLM compression technologies across the industry. Companies that can effectively miniaturize AI without sacrificing performance will likely gain a competitive edge, potentially influencing the development roadmaps of major tech players and the adoption strategies of AI-powered applications.