The scourge of x86 emulation

Emulating x86 software on ARM now runs demonstrably faster, without requiring emulator developers to change their code.

The FEX-Emu project, an x86 and x86-64 emulator for Linux, faces significant performance challenges due to emulating the x86 Total Store Ordering (TSO) memory model on ARM's weaker memory consistency model. X86-TSO ensures that memory stores are visible to all processors and loads respect prior stores, a strict model that aligns with programmer intuition. ARM, conversely, uses a relaxed memory model for efficiency, where stores are not immediately coherent across processors and loads may not see the latest writes. To emulate x86-TSO on ARMv8.0-a, FEX-Emu previously relied on load-acquire and store-release instructions, which, while providing necessary ordering, incur substantial performance overhead, as evidenced by microbenchmarks showing hindered performance on several ARM CPUs. Newer ARM architectures, specifically ARMv8.3 and later, introduce Release Consistency processor consistent (RCpc) and load-LRCPC instructions, which more closely align with x86-TSO requirements and significantly improve emulation performance. Apple's M1 processors offer a unique solution by directly supporting the x86-TSO memory model at the hardware level, allowing regular ARM load/store instructions to behave as required for x86 emulation with minimal performance impact.

The FEX-Emu project's struggle with x86-TSO emulation highlights a fundamental divergence in CPU design philosophies: x86 prioritizes predictable programmer experience through strict memory ordering, while ARM optimizes for efficiency with relaxed consistency. This disparity necessitates complex workarounds, such as the load-acquire and store-release instructions, which prove costly on older ARM hardware. The emergence of ARMv8.3's RCpc model and LRCPC instructions represents a significant architectural adaptation, specifically catering to the demands of x86 emulation and effectively resolving the performance bottleneck on newer ARM platforms. This evolution suggests a growing recognition within the ARM ecosystem of the need to support high-performance legacy code execution, essential for broader adoption and cross-platform compatibility.

Apple's direct hardware implementation of x86-TSO on its M1 processors offers a compelling alternative, bypassing the need for emulation-specific instructions and achieving near-native performance. This approach underscores a strategic decision by Apple to facilitate a seamless transition to ARM for x86-based software, potentially setting a precedent for other chip manufacturers. While FEX-Emu's developers observe minimal performance degradation on ARM code when TSO mode is active, the long-term implications for hybrid applications or systems where strict memory ordering is not universally required remain an area to watch. The success of both architectural adaptations—ARMv8.3's specialized instructions and Apple's TSO hardware toggle—indicates a clear market demand for efficient x86 emulation, driving innovation in memory model handling across the semiconductor industry.