Baidu released ERNIE 5.5 Flash Base

ERNIE 5.5 Flash Base is now available as an open model checkpoint, allowing developers to build upon its capabilities without incurring the cost of large-scale pre-training.

Baidu has released ERNIE 5.5 Flash Base, a sparse Mixture-of-Experts (MoE) large language model. This base model features 280 billion total parameters, with 14 billion activated per token. It was pre-trained on 33.5 trillion tokens and supports a context window of up to 128K during pre-training, extending to 1 million tokens for fine-tuning and inference. The model's architecture incorporates innovations such as Latent Gated Sparse Attention, Multi-Router Latent MoE, Manifold-Constrained Hyper-Connections, and a MultiMax Output Head. Baidu emphasizes efficiency as a core design goal, balancing model capacity, activated compute, and long-context costs. The release provides open weights from the end of pre-training, intended as a starting point for developers to build upon for various domain adaptations, reasoning tasks, and agentic applications.

The release of ERNIE 5.5 Flash Base signals a continued industry push toward more efficient and adaptable large language models. By providing an open base model checkpoint, Baidu aims to democratize access to advanced AI capabilities, enabling researchers and developers to focus resources on specific downstream tasks and innovations rather than foundational pre-training. This strategy could accelerate the development of specialized AI applications across various domains.

Key architectural innovations like Latent Gated Sparse Attention and Multi-Router Latent MoE are designed to manage computational costs associated with long contexts and large parameter counts. These efficiency-driven design choices suggest a market trend where performance gains must be balanced with practical deployment and training affordability. Developers leveraging this model can expect to explore more complex reasoning and agentic behaviors with reduced inference and fine-tuning overhead.