Revolutionizing Mac-Based AI Development
PyTorch 2.15 has arrived, marking a significant milestone for developers utilizing Apple silicon hardware. This release, driven by extensive contributions from the Hugging Face team and the PyTorch community, shifts the MPS (Metal Performance Shaders) backend away from generic graph-based execution toward highly optimized, custom Metal kernels. This architectural pivot directly addresses performance bottlenecks, memory management issues, and feature parity gaps that previously hindered Mac-based machine learning workflows.
For practitioners working with local Large Language Models (LLMs), video generation, or complex scientific computing on macOS 15 and above, PyTorch 2.15 offers a transformative experience. By replacing indirect execution paths with native Metal operations, the framework now maximizes the throughput of M-series chips, turning Apple hardware into an even more formidable platform for cutting-edge AI development.
Breakthroughs in LLM Inference
The update brings substantial speed improvements for the decoding phase of transformer models. By optimizing linear layers and integrating native support for grouped matrix multiplications, 2.15 dramatically boosts token throughput. Specifically, Mixture-of-Experts (MoE) models—which are notoriously difficult to run efficiently—see decoding speeds increase between 5 and 6.5 times compared to version 2.14. A model like Qwen3.6-35B-A3B jumps from roughly 9 tokens per second to over 60 tokens per second on an M3 Ultra, making real-time local interaction a reality for much larger models than previously possible.
Complementing these gains, the framework now includes optimized softmax and log_softmax kernels. These are critical for the final stages of token sampling, with tests showing an 8x speedup for large vocabulary sizes (e.g., [1, 128256] shapes). This optimization ensures that the sampling stage no longer acts as a throttle during high-speed text generation.
Massive Memory Efficiency for Creative Workflows
Perhaps the most significant technical achievement in this release is the drastic reduction in memory footprint for video generation models. In earlier versions, processes like the tiled Variational Autoencoder (VAE) decoding—central to models such as Wan2.2—suffered from inefficient padding operations that caused driver memory usage to balloon to upwards of 270 GiB. By moving constant padding to Metal-native kernels, PyTorch 2.15 has reduced the peak driver memory usage to just 8 GiB for the same workload.
This memory efficiency is a game-changer for content creators and researchers using tools like ComfyUI, as it prevents system crashes and enables the execution of heavy generative workflows on consumer-grade unified memory configurations that previously lacked the overhead to handle such massive memory spikes.
Expanding Capabilities with FP8 and Linear Algebra
PyTorch 2.15 introduces native support for FP8 (float8_e4m3fn) tensors on MPS. This format halves the memory storage requirements compared to traditional BF16 or FP16, enabling more aggressive model quantization and larger context windows within the same memory constraints. The inclusion of scaled matrix multiplication (F.scaled_mm) ensures these FP8 tensors can be processed directly on the GPU, bridging a critical feature gap with CUDA-based workflows.
Furthermore, the linear algebra suite has been expanded, with complex-valued LU factorization and triangular solves now running natively on the GPU. Small matrix operations, essential for computer vision and camera geometry, have also been streamlined through dedicated kernels, ensuring that specialized scientific tasks run with minimal CPU-to-GPU synchronization latency.
Why it Matters
- Performance: Massive reductions in latency for LLM token decoding and complex linear algebra operations.
- Efficiency: Video generation tasks that once required massive memory overhead now fit comfortably on standard Apple silicon configurations.
- Compatibility: Expanded operator support ensures that modern generative AI tools and libraries run reliably without the need for manual workarounds.
- Control: Introduction of explicit MPS stream management allows developers to fine-tune execution and memory allocation for complex, parallelized pipelines.










