Vercel added Hy4 Preview, Tencent's model, to its AI Gateway. It's an open source Mixture-of-Experts with 770B total parameters, 49B active per token, and a 1M context window, aimed at long-horizon coding tasks, document analysis and scientific reasoning. The announcement is in Vercel's changelog, the model was released by Tencent, and Simon Willison has already covered it.

Why it matters

The technical news is the model. The structural news is something else: switching inference providers has become editing a string. To use Hy4 you just put tencent/hy4-preview in the AI SDK's model field, and to hook it up to a coding agent you run vercel ai-gateway coding-agents setup and pick it from Claude Code, Codex, OpenCode, Cursor or Pi.

That turns the model into a configuration parameter, not an architectural decision. And there's the signal I care about: if the cost of switching models trends to zero, your product's differentiator stops being which model you use. It becomes how well you measure which one serves you. The team that knows how to evaluate wins; the one that picks based on an announcement benchmark loses money without noticing.

There's a second detail that isn't marketing: Vercel says it mirrors provider pricing with no markup and no platform fee on inference, even with Bring Your Own Key. If that holds, the gateway competes on routing, observability and failover, not on margin over tokens. That's a healthier business model for whoever is integrating.

What changes in practice

The concrete part for someone who already has an LLM app in production:

  • The model layer becomes configuration, not code. If you still have the model name hardcoded across three different files, this is the moment to pull it out into an environment variable with an explicit default.
  • MoE models change how you do cost math in your head. 770B total with 49B active per token means the total size contributes the quality and the active fraction contributes the inference cost. Don't compare models by total parameters: you're comparing apples to oranges.
  • 1M of context enables flows that used to require RAG. Dropping a whole repo or a document corpus into the window stops being a fantasy. Careful: enabling isn't recommending, and I come back to this below.
  • You need your own evaluation set before touching anything. Without it, switching models is gambling.

The minimum way to get this ready is this, and it takes no more than twenty minutes:

// config/models.ts
export const MODEL = process.env.LLM_MODEL ?? 'tencent/hy4-preview'

// A single place where the model name lives.
// Switching providers = changing an env var, not a code deploy.

And then, what really matters: a file of real cases from your domain, with the expected output, run against the two or three candidate models. Twenty cases of your own are worth more than any published comparison table.

When NOT to use it

Here's the honest part, because the announcement doesn't say it.

It's a preview. The word is in the name. A preview can change behavior, tokenizer, availability or pricing without useful notice. Don't put it in the critical path of a product that bills, unless you have failover configured and tested against another model.

There are no benchmarks in the material I have. The changelog describes capabilities and target areas, not measured results. Anyone telling you today that Hy4 performs better or worse than their favorite alternative is extrapolating. I'm not going to do that.

1M of context is not 1M of useful attention. It's the most expensive trap of this generation of models. A million tokens fitting doesn't guarantee the model reasons as well about token 900,000 as it does about token 5,000. Before throwing your RAG pipeline in the trash, measure retrieval at intermediate positions with your own documents. And remember the cost: filling the window on every request is a recurring bill that good retrieval saves you.

A gateway adds a network hop. You gain routing, failover, reporting and per-API-key budgets. You pay with extra latency and one more dependency in your diagram. If your case is a single call to a single provider at low volume, the abstraction doesn't pay for itself.

Open source doesn't mean you can run it. A 770B MoE doesn't fit on your laptop or on your cloud's cheap instance. Open weights are valuable at the ecosystem and audit level, but operationally you're still consuming someone's API. Don't confuse licensing with infrastructure sovereignty.

Data residency. If you work with regulated information, adding a new provider is a legal conversation before a technical one. Vercel mentions Zero Data Retention support, but you verify that by reading the terms, not a changelog.

What I'd do today

I'd add it as a candidate in my evaluation harness this week, on the long-context cases that hurt today: analysis of lengthy documents and refactors that touch many files. I'd measure accuracy, latency and real cost per completed task, not per token.

It's not going to production yet. A preview with no public benchmarks is a promising-looking experiment, and experiments go behind a flag, with a stable model as fallback. If in two weeks my harness numbers justify it, I promote it. If not, I've already gained something anyway: the harness is built for the next announcement, which will arrive sooner than you think.