Every few weeks a model drops that makes me check my own assumptions about running huge AI locally. Tencent’s newly open-sourced Hy4 preview is one of those. It’s a 770-billion-parameter Mixture-of-Experts model with just 49B active per token, a 1-million-token context window, and weights released under Apache 2.0. That combination — big, sparse, and permissive — is exactly what makes it both exciting and practical.
The commands and figures below all come from Tencent’s official release announcement, the Hy4 preview model card on Hugging Face, and the OpenRouter model listing — cross-checked against the live APIs so you’re not getting anything that doesn’t actually exist.
I spent the morning poking at it from every angle a developer actually would: what’s on Hugging Face, how OpenRouter serves it, what the config tells us, and what the official vLLM and SGLang recipes look like. This guide walks you through what I verified live so you can decide between the two realistic paths: calling it through an API, or pulling the FP8 weights and running it on your own cluster. No gatekeeping — if you don’t have eight H100s lying around, the API route gets you there too, and I’ll be honest about the trade-offs.
What Exactly Is Tencent Hy4 Preview?
Hy4 preview is the newest generation of Tencent’s Hunyuan model family, and it’s a big architectural step from the Hy3 releases earlier this year. The naming passes a real milestone: it’s the first Hunyuan flagship to ship an official FP8-quantized build alongside the BF16 weights, which is what makes self-hosting at least thinkable on gear that isn’t purely a server-rack flex.
The specs I pulled directly from the model card and config.json are worth reading closely:
- Architecture: Mixture-of-Experts (MoE), 78 layers
- Total parameters: 770B (49B activated per token)
- MoE config: 256 routed experts + 1 shared expert, top-8 activated per token
- Attention: Gated DeepSeek Sparse Attention with IndexCache
- Context length: 1M tokens
- Hidden size: 6144; 64 attention heads
- Vocabulary: 120,832
- License: Apache 2.0
A 770B-parameter model that only activates 49B per token is the whole trick. Most of the model stays dormant on any given request, which keeps inference cost far below what a dense model of that size would demand. That’s the same reason I keep coming back to the open-weight landscape — the economics of sparse MoE models are quietly redefining what “frontier open-source” means, and I wrote about that trend in why open-weight AI has become the Valley’s hottest acquisition target. Hy4 is the latest proof point.
How to Try It Right Now: The OpenRouter Path
The fastest way to run Hy4 preview starts with zero infrastructure. OpenRouter picked it up the same week it launched, and it’s served under the id tencent/hy4-preview. I verified this live against their model catalog — the entry shows the full 1,048,576-token context and a serving endpoint tagged tencent/fp8.
Because OpenRouter API is OpenAI-compatible, any code you already use for GPT or Claude swaps over by changing the base URL and model name. Here’s a minimal call:
curl --request POST \
--url https://openrouter.ai/api/v1/chat/completions \
--header "Authorization: Bearer $OPENROUTER_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"model": "tencent/hy4-preview",
"messages": [
{"role": "user", "content": "Refactor this function to be async and explain the trade-offs in three sentences."}
],
"temperature": 0.9,
"top_p": 1.0
}'This is a drop-in replacement, so it slots straight into existing agent loops, chat UIs, and CLI tools. If you’re already juggling multiple providers, one of the smartest things you can do is push Hy4 into a model router built on LiteLLM alongside your existing endpoints, so the switch is a config change instead of a code change.
Pricing You Can Actually Budget Against
The API price is set by Tencent and mirrors exactly across OpenRouter: $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million tokens for cache reads. That’s the same number Tencent lists on its own release page, which is a nice sign there’s no reseller markup hiding in the middle tier.
For context on a typical coding workload, a prompt with a few thousand tokens plus a couple hundred tokens of output lands well under a cent. The real advantage shows on big-context work — being able to hand a model an entire repository or a long document in one pass and only pay for what it actually processes is a different calculation than a sliding-window model that re-tokens everything.
Self-Hosting It Yourself: The vLLM and SGLang Path
If you’d rather own the inference — or you want to avoid sending code to a third party — the weights are on Hugging Face under tencent/Hy4-preview (BF16) and tencent/Hy4-preview-FP8 (the quantized build). I confirmed both repos are live; the FP8 variant is the realistic one to pull unless you have truly serious hardware, because it cuts the memory footprint meaningfully.
Before you get excited, the hardware reality: this is not an Ollama-on-your-laptop situation. A 770B MoE model like this wants eight GPUs in tensor parallel. For reference on how the other end of the spectrum works — squeezing a big model onto one modest card — my AirLLM tutorial on running a 70B model on a 4GB GPU is the opposite extreme, and it helps calibrate expectations about what quantized single-GPU inference can and can’t do.
The official vLLM recipe uses a prebuilt image and an eight-way tensor-parallel split:
docker run --gpus all \
-p 8000:8000 \
--ipc=host \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:hy4-preview tencent/Hy4-preview-FP8 \
--tensor-parallel-size 8 \
--speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
--attention-backend FLASHMLA_SPARSE \
--tool-call-parser hy_v4 \
--reasoning-parser hy_v4 \
--enable-auto-tool-choice \
--port 8000 \
--served-model-name hy4-previewOnce the server is up on localhost:8000, it exposes an OpenAI-compatible endpoint, and the official quickstart calls it with the Python OpenAI SDK:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="hy4-preview",
messages=[
{"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
],
temperature=0.9,
top_p=1.0,
)
print(response.choices[0].message.content)SGLang is the lighter alternative — same FP8 model, eight-way tensor parallel, and the prebuilt lmsysorg/sglang:hy4-preview image handles multi-arch (x86 and Arm), which matters if your nodes aren’t all the same silicon.
Reasoning Mode: The One Setting You Should Know
Hy4 preview defaults to a “high” reasoning mode — deep chain-of-thought. That’s right for math, code, and hard reasoning tasks, but it’s overkill if you just want a quick factual answer. The model card specifies how to bypass it: pass reasoning_effort: "no_think" in the extra request body to get a direct response without the long internal monologue. That single toggle is the difference between a snappy tool and one that feels like it’s thinking for forty seconds.
Tencent flags this as a known limitation of the preview, by the way — the model “spends longer than necessary reasoning through complex tasks” and has a tendency to over-verify its own work. So if you hit a response that feels oddly cautious, that’s the model doing exactly what the team admits it does. Budget for it in your prompts.
Eight Machines or One API Key: Making the Call
So which path should you take? It comes down to the same question I ask with any self-hosted stack: do you want to learn the plumbing or ship the product?
The API route costs nothing to start, needs one environment variable, and — because OpenRouter’s surface is OpenAI-shaped — drops into software you already run. For most developers and small teams, that’s the rational default. It’s also the honest answer when you don’t have a GPU cluster already doing things; buying eight cards to run one open model rarely pencils out.
The self-host route is for when you have the hardware, the data-control requirements, or an existing inference fleet you want to extend. If you’re already comfortable standing up local models with Ollama, the jump to a vLLM container on a real GPU box is a natural next step — same OpenAI-compatible idea, just far more horsepower behind it.
What I Left You With
Here’s my takeaway after an hour with Hy4 preview: the model itself is a legitimately interesting milestone — a 770B open-weight MoE with a million-token window and permissive licensing — but the real story for most readers is that you can try it today without any new hardware.
Start with the OpenRouter call and a pair of questions you genuinely care about. Test the coding output against your current model of choice. Flip the reasoning_effort setting and feel the difference between deep thought and a direct answer. If it earns a spot in your workflow, that’s when you think about whether the end-to-end control of the self-hosted route is worth spinning up the GPUs.
The open-source frontier keeps raising its own bar, and sparse MoE is the reason why. Whether Hy4 preview is your entry point or just another benchmark, it’s worth fifteen minutes to find out.