Signal

Inkling 2 Mini

First reported by Thinkingmachines ·

The signal ●●○○ Compiled by AI from Thinkingmachines and Hacker News
Why you might care

Powerful AI models that fit on your laptop are now available, and they are even better at reasoning and coding tasks.

What happened

Thinking Machines Lab has released Inkling 2 Mini, a new open-weights language model designed for local execution on consumer hardware. This model utilizes a "looped transformer" architecture, running its middle layers three times per token to enhance reasoning capabilities, particularly for tasks like agentic coding. On benchmarks such as DeepSWE v1.1 and Vals Index v2.2, Inkling 2 Mini outperforms other open-weights models with under 100 billion parameters. It achieves a theoretical tokens per second ceiling of 614 GB/s. With a total parameter count of 26.4 billion and 3.4 billion active parameters, its memory footprint is approximately 20GB, making it suitable for laptops. The lab demonstrated its capability by having it build a flight simulator offline on a MacBook Pro. Inkling 2 Mini is available for inference and customization on Tinker and Hugging Face.

What it means

Inkling 2 Mini's "looped transformer" architecture, where middle layers are processed multiple times per token, represents a significant advancement in optimizing reasoning without drastically increasing model size or computational cost. This design choice allows the model to apply more complex thought processes, leading to better performance on tasks requiring sequential reasoning and agentic capabilities, and it has proven effective even when the model is run offline on consumer-grade hardware.

The model's strong performance on coding and agentic benchmarks, coupled with its ability to leverage tools and search efficiently, suggests a future where highly capable AI assistants can be deployed locally, reducing reliance on cloud infrastructure and offering greater privacy and accessibility. The focus on monitorability, enabled by the looped architecture, also addresses growing concerns about understanding and controlling AI behavior.

AI-written summary. May contain errors.

Inkling