I’ve been running local AI models on various hardware for a while now, but I haven’t seen anything quite like this. Meta just launched Muse Spark Open, an 816-billion-parameter open-weight model that runs entirely on Apple silicon — and it’s the first frontier-class AI model I’ve seen that actually treats the Mac as a serious inference platform, not just a client for cloud APIs.
Apple and Meta teamed up to make this happen. The result is a mixture-of-experts model with 816 billion total parameters and 42 billion active parameters per inference pass. The “open-weight” part means you can download the weights and run the model locally — no cloud subscription, no API key, no data leaving your hardware. The catch? You need some serious Mac hardware.
What Muse Spark Open Actually Is
Muse Spark Open is a multimodal mixture-of-experts model — it handles text, images, audio, and video. Meta designed it for complex coding tasks and agentic workflows, and they claim it’s the most capable open-weight model released in the United States to date.
The model comes in two formats: BF16 (full precision, requires about 2TB of memory) and NVFP4 (quantized, about 1TB of memory). For the Mac Studio setup, you’ll be using the NVFP4 variant.
The Meta AI research page describes it as “natively multimodal and built for complex coding tasks and agentic labor.” The Hugging Face model card has the technical details and download links.
The Hardware Requirement: Mac Studio M5 Ultra
This isn’t a model you can run on a MacBook Air. Meta and Apple specify two configurations:
- Two-computer setup: Two Mac Studio machines with M5 Ultra chips and 512GB of unified memory each
- Four-computer setup: Four Mac Studio machines with M5 Ultra chips and 256GB of unified memory each
The current Mac Studio with M5 Ultra and 256GB of memory starts at $10,799. A 512GB version is coming later this month. So you’re looking at roughly $38,000 for the four-machine setup. This is firmly in “I have a budget for serious AI hardware” territory.
Why so much memory? The model weights take up about 537GB per machine in NVFP4 format. With the distributed inference setup, each Mac holds a portion of the model, and they share the workload over Thunderbolt 5.
Software Stack: MLX and JACCL
Apple’s MLX framework is the key piece. It’s an open-source machine learning library built specifically for Apple silicon, and it has a distributed inference feature that lets you split a model across multiple Macs.
The magic sauce is JACCL — Apple’s open-source collective communication library. It uses RDMA (Remote Direct Memory Access) over Thunderbolt 5 to let the Macs talk to each other with very low latency. This is what makes the multi-Mac setup actually work: the model isn’t just split across machines, it’s actively coordinated during inference.
The Apple Machine Learning research page has the full technical breakdown. The key requirement is macOS 26.2 or later, which introduced the low-latency RDMA driver over Thunderbolt 5. You’ll also need MLX 0.30.1 or later.
Setting Up the Cluster
Here’s the practical workflow for getting Muse Spark Open running across multiple Macs. I’m basing this on the documented deployment process from Meta and Apple.
1. Prepare the Hardware
Connect the Macs with Thunderbolt 5 cables. The documented setup uses a mesh topology — every Mac connected to every other Mac. For the two-Mac setup, you use two Thunderbolt cables between them to double the link bandwidth.
Enable RDMA on each Mac: open Settings, search for “RDMA,” and click “Enable RDMA over Thunderbolt.” Reboot each machine afterward.
2. Install the Software
On each Mac, install MLX and the MLX-LM language model package:
pip install -U mlx-lm
This pulls in MLX as a dependency. You’ll also need Hugging Face Hub to download the model:
pip install -U huggingface_hub
3. Download the Model
The model weights are about 537GB per machine. You’ll need to download the NVFP4 checkpoint on each Mac. The Hugging Face CLI makes this straightforward:
huggingface-cli download meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --local-dir ./muse-spark-open
Make sure you have at least 600GB of free storage per machine — the download plus working space adds up fast.
4. Configure the Hostfile
The mlx.launch command reads a JSON hostfile that describes your cluster. For the two-Mac setup, your hostfile would look something like this:
{
"hosts": [
{"host": "mac1.local", "port": 22},
{"host": "mac2.local", "port": 22}
]
}
Replace the hostnames with your Macs’ actual network names. Make sure SSH access works between all machines.
5. Launch the Distributed Server
Run this on the primary Mac:
mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080
This starts the MLX language model server on every Mac in the cluster and serves an OpenAI-compatible API on port 8080. The --backend jaccl flag enables the distributed communication layer.
Running Inference
Once the server is running, you can hit it with any OpenAI-compatible client. The server exposes the standard chat completions API.
For a quick test, you can use curl:
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "muse-spark-open",
"messages": [{"role": "user", "content": "Explain how transformers work"}]
}'
Or use it with your favorite AI coding tool — most of them let you point to a custom OpenAI-compatible endpoint. If you’ve been following my earlier guide on how to run local AI for daily tasks, the workflow is similar, just with a much bigger model.
Performance Expectations
Apple reports more than 36 generated tokens per second for Muse Spark Open on a four-Mac Studio cluster. That’s well above average human reading speed — fast enough for real-time coding assistance.
For context, that’s significantly faster than most consumer GPUs running smaller models. The unified memory architecture on M5 Ultra is doing heavy lifting here: the GPU can access up to 512GB of memory directly, without the bandwidth bottlenecks of traditional GPU memory setups.
If you’re curious about how this compares to other AI tools I’ve covered, the practical guide to using AI tools effectively has a broader workflow perspective.
Practical Use Cases
What can you actually do with an 816B open-weight model running locally?
Agentic coding: The model is designed for complex coding tasks. You can point it at your codebase and have it make changes, refactor functions, or debug issues — all locally, with your code never leaving your machine.
Multimodal analysis: Because it handles images, audio, and video, you can feed it screenshots, diagrams, or video content for analysis. This is useful for design review, content moderation, or accessibility auditing.
Data privacy: For organizations that can’t send data to cloud APIs, this is a genuine alternative. The model runs entirely on hardware you control. Healthcare, finance, legal — any industry with strict data residency requirements.
Research and fine-tuning: The open-weight license allows limited commercial use. You can fine-tune the model for specific domains or tasks using PyTorch and TorchTitan, as Meta documents.
The Cost Equation
Let’s be honest about the economics. A two-Mac setup with 512GB machines will cost roughly $22,000. The four-Mac setup with 256GB machines is about $38,000. Plus you need the Thunderbolt 5 cables, which aren’t cheap.
Compare that to cloud API pricing. At current rates, processing millions of tokens through an API like OpenRouter or Fireworks would cost a fraction of that — at least until you’re processing enough volume to justify the capital expense.
The break-even point depends heavily on your usage. For a team of developers running agentic coding workflows all day, the local setup could pay for itself within a year. For occasional use, it’s hard to justify.
But there’s a non-financial angle too: data sovereignty. If you’re working with sensitive code, proprietary data, or regulated information, the ability to run a frontier-class model without sending anything to a third party has real value that doesn’t show up on a spreadsheet.
Getting the Most Out of the Setup
A few practical tips from what I’ve learned running distributed systems:
- Cooling matters. These Macs will be running at sustained load for hours. Make sure they’re in a well-ventilated area, not stacked on top of each other in a closed cabinet.
- Storage speed is critical. The model weights are read repeatedly during inference. Use the fastest SSD configuration possible — external Thunderbolt SSDs work, but internal storage is better.
- Network topology matters. The Thunderbolt 5 mesh is the backbone of the cluster. Don’t daisy-chain; connect every Mac directly to every other Mac for maximum bandwidth.
- Start with the two-Mac setup. It’s cheaper to set up and easier to debug. You can always add more machines later.
The Bigger Picture
This launch signals something important about the AI industry. For years, the story has been that serious AI requires serious GPU infrastructure — racks of H100s in data centers, or at minimum a high-end consumer GPU with 24GB of VRAM. Apple and Meta are showing that a different path is possible: using the unified memory architecture of Apple silicon to run models that previously required data center hardware.
It’s not for everyone. The cost is high, the setup is complex, and it’s limited to Apple hardware. But for teams that already use Macs and need local AI inference, this is a genuine option that didn’t exist a year ago.
If you’re interested in the broader landscape of AI model development, Meta’s been on a roll lately — their Muse Wonder model was their first attempt at a frontier-class model, and Muse Spark Open is the open-weight follow-up that makes the technology accessible to anyone willing to invest in the hardware.
Whether this becomes a mainstream deployment model or remains a niche option for well-funded teams, it’s a reminder that the AI hardware landscape is more diverse than the “GPUs or bust” narrative suggests. And for those of us who like to tinker with new technology, it’s a pretty exciting development.