Artificial IntelligenceTechnical Deep Dive

Ai2 Unveils Olmo-core 3: A New Open Standard for Trillion-Parameter AI Training

Published
EElectricBuzz Editorial Team
Ai2 Unveils Olmo-core 3: A New Open Standard for Trillion-Parameter AI Training
3 min read518 wordsElectricBuzz Editorial Team

The Gist

“The Allen Institute for AI is tackling the scaling challenges of Mixture-of-Experts models with a new, highly efficient training infrastructure.”

Revolutionizing Mixture-of-Experts Training

The Allen Institute for AI (Ai2) has officially launched Olmo-core 3, a significant evolution in training infrastructure designed specifically for large-scale Mixture-of-Experts (MoE) models. As AI developers look to move beyond dense model architectures, MoEs have emerged as a leading solution for scaling parameter counts without linearly increasing computational costs. However, managing the complex routing and memory overhead associated with these models remains a hurdle. Olmo-core 3 addresses these bottlenecks, enabling developers to push model sizes into the trillion-parameter territory while maintaining high computational efficiency.

By shifting from previous fully sharded data parallelism (FSDP) implementations to a refined approach based on distributed data parallelism (DDP), the team has achieved substantial performance gains. Preliminary benchmarks conducted on NVIDIA B300 GPUs demonstrated a 2.7x increase in throughput compared to their earlier frameworks. This shift ensures that specialized experts remain resident on specific GPUs, drastically reducing the latency associated with repeatedly gathering and resharding model weights during training.

Advanced Optimization Techniques

At the heart of Olmo-core 3 is a sophisticated suite of optimization strategies designed to maximize GPU utilization. To distribute massive MoEs across hardware clusters, the framework employs three primary pillars: expert parallelism, pipeline parallelism, and distributed optimization. Expert parallelism partitions the expert pool across nodes, while pipeline parallelism breaks model layers into stages to keep memory requirements manageable. The distributed optimizer ensures that optimizer state data is spread across the GPU cluster rather than being duplicated in its entirety on every chip.

The system also introduces low-level refinements to streamline data movement and computation. Features such as GPU-resident routing metadata allow the CPU to queue tasks without waiting for round-trip data transfers, while grouped GEMM operations combine numerous smaller expert computations into larger, more efficient blocks. Additionally, support for the MXFP8 number format offers a significant efficiency boost; benchmarks indicated a 21% increase in training throughput compared to standard BF16 precision, while simultaneously reducing peak memory usage by nearly 8%. These combined techniques give researchers granular control over the complex trade-offs between speed, memory footprint, and precision.

Pushing Toward the Trillion-Parameter Horizon

The practical potential of Olmo-core 3 is evidenced by its performance at the extreme scale. During internal testing, the team successfully benchmarked the infrastructure with models containing up to 1.2 trillion parameters, maintaining an impressive throughput of 858 TFLOP/s/GPU. These tests were not just theoretical; they provided critical insights into the real-world mechanics of training large models, such as the avoidance of "token gerrymandering"—a failure mode where routing seems balanced but degrades in quality—and the nuances of overlapping communication with computation.

As an open-source project, Olmo-core 3 represents more than just a tool for the Allen Institute; it serves as a foundation for the entire research community. By providing transparent infrastructure, Ai2 aims to democratize the development of highly capable AI models. The framework is designed to be extensible, allowing developers to adapt it to diverse hardware environments and experiment with different routing or parallelism strategies. This release underscores the belief that truly open AI requires not only accessible model weights but also full transparency regarding the infrastructure and methodologies used to train them.

SPONSORED
The 5 Best Over-Ear ANC Headphones of 2026, Tested & Ranked
Editor's Pick Guide
92/100
Tech & Gadgets•12 min read

The 5 Best Over-Ear ANC Headphones of 2026, Tested & Ranked

We locked five over-ear ANC picks for 2026 — Sony WH-1000XM6, Bose QuietComfort Ultra 2, Soundcore Space One, Sennheiser Momentum 5, and Apple AirPods Max 2 — then stress-tested them on lab metrics, long-term owner truth, and live street prices.

Related Stories

Semantically matched articles, ranked by topic overlap and freshness.

Meta’s Muse Charm: A $300 Pocket-Sized AI Companion
Artificial Intelligence

Meta’s Muse Charm: A $300 Pocket-Sized AI Companion

Meta is reportedly readying its 'Muse Charm' AI wearable for a $300 price point, aiming to turn its agentic AI software into a physical, keychain-sized companion.

Apple Prepares to Enter the AI Agent Race: A New Vision for macOS
Artificial Intelligence

Apple Prepares to Enter the AI Agent Race: A New Vision for macOS

Apple is reportedly developing a sophisticated AI agent capable of operating your Mac, marking a strategic pivot toward proactive, multi-app automation.

Nvidia in Flux: The Unfolding Departure of Jensen Huang
Artificial Intelligence

Nvidia in Flux: The Unfolding Departure of Jensen Huang

A deep dive into the corporate crisis at Nvidia following the sudden resignation of longtime CEO Jensen Huang amid national security concerns and personal controversies.

Kakao Founder Cleared of Stock Manipulation Charges
Artificial Intelligence

Kakao Founder Cleared of Stock Manipulation Charges

A South Korean appeals court has upheld the acquittal of Kakao Corp. founder Brian Kim, removing a significant legal hurdle for the tech giant.

Supply Chain Disruptor Everful Secures $48.5M to Automate Global Sourcing
Artificial Intelligence

Supply Chain Disruptor Everful Secures $48.5M to Automate Global Sourcing

Everful is bridging the gap between small retailers and Chinese manufacturing through an AI-driven, end-to-end supply chain platform.

Anthropic’s IPO: A Stand for Controlled AI Evolution
Artificial Intelligence

Anthropic’s IPO: A Stand for Controlled AI Evolution

As Anthropic makes its historic public market debut, CEO Dario Amodei reinforces a firm commitment to safety-first development, vowing to prioritize human control over rapid deployment.

Meta’s Muse AI Agent Under Fire Over Privacy Breaches and Cross-Account Data Leaks
Artificial Intelligence

Meta’s Muse AI Agent Under Fire Over Privacy Breaches and Cross-Account Data Leaks

Internal reports suggest that Meta's rapidly growing AI agent, Muse, is inadvertently surfacing private user data and mixing up personal communications.

Anthropic Makes Historic Nasdaq Debut, Valuation Soars Past $2 Trillion
Artificial Intelligence

Anthropic Makes Historic Nasdaq Debut, Valuation Soars Past $2 Trillion

AI powerhouse Anthropic has officially hit the public markets, opening at a staggering 33% premium over its already elevated IPO price.