Meta Unveils Muse Spark Open: An 816 B‑Parameter Open‑Weight Multimodal MoE Model

Meta has released Muse Spark Open, an open‑weight frontier model that pushes multimodal AI beyond text alone. With roughly 816 billion total parameters and about 42 billion active per token, the model ingests text, images, audio, and video through a mixture‑of‑experts (MoE) design. Each of its 62 layers contains 256 experts, yet only 8 are routed per token, delivering high capacity while keeping compute efficient.

Agentic Coding and Multimodal Input

Built for agentic coding, Muse Spark Open can read entire codebases, generate and debug scripts, and invoke tools autonomously. Its multimodal pipeline lets the same model consume a screenshot, a voice note, or a screen recording while it works on code, enabling developers to ask questions across different input types through a unified interface.

The model was trained on real developer workflows such as multi‑file refactoring and extended debugging sessions. It features a context window of up to one million tokens, checkpoint formats in BF16 and NVFP4, and shares the tokenizer and vocabulary with the smaller Muse Glimmer release. Input embeddings and the output layer are separate matrices of about 1.66 B parameters each.

Why It Matters for GPU / AI Infrastructure

For infrastructure planners, the 816 B parameter footprint and MoE routing demand substantial GPU memory—equivalent to an 816 B dense model—while delivering lower per‑token compute. NVFP4 checkpoints further compress storage needs, but inference requires careful load‑balancing and high‑throughput interconnect fabrics. Companies like AIGPU that provide scalable GPU clouds can leverage these capabilities to offer on‑demand clusters optimized for large‑scale MoE workloads.

Muse Spark Open is available on GitHub for fine‑tuning and deployment. Its open‑weight nature encourages community contributions and accelerates research into multimodal agentic systems, setting a new benchmark for transparency and capability in open models.