Elorian’s first AI model wants to reason with images, not just about them

Visual AI models can now analyze images and videos with more depth, making it easier to understand complex visual data without manual intervention.

Elorian, a Palo Alto-based AI startup, has launched its first model, Glimpse. This model is designed to "reason with images" by repeatedly referencing the source image or video during its analysis, rather than just processing it once into tokens. Glimpse can crop into regions, pull specific frames, or run code to measure and compare visual elements, a process Elorian calls a "visual scratch space." This approach aims to overcome limitations in current AI models that struggle with visual languages and iconography, as illustrated by an inability to interpret a board game's status. Elorian claims Glimpse was trained on a few hundred GPUs and is smaller than competing frontier systems, offering competitive performance on tasks like scientific chart reasoning and video captioning at a specified price point. The company, which raised $55 million in seed funding, is targeting companies with large video datasets, such as robotics firms and media companies, with plans to eventually enable AI to control robots.

Elorian's Glimpse model represents a significant step towards AI that truly "sees" by employing a persistent reference to visual data during reasoning. This contrasts with existing multimodal models that tokenize images once, limiting their ability to interpret nuanced visual information like game states or assembly instructions. The "visual scratch space" mechanism allows for iterative refinement and deeper understanding, potentially unlocking new applications in fields reliant on visual interpretation.

The company's claim of efficient training and smaller model size, coupled with competitive performance benchmarks, suggests a potential shift in the cost-effectiveness of advanced visual reasoning. If successful, this could democratize access to sophisticated visual AI for a wider range of businesses, particularly those with extensive video archives. The future goal of enabling AI to directly control robots based on visual scene understanding further signals a move towards more embodied AI agents.