Latest in AI
Tencent Hy4 Open Weights: Plan Capacity Before Running the Flagship
Hy4's 49B active parameters do not make it a 49B deployment. Choose the right checkpoint, serving image and evaluation before allocating GPUs.
Tencent's Hy4 flagship now has downloadable weights and deployment examples, giving teams an alternative to evaluating it only through a hosted API. The useful question is no longer just whether its answers are good. It is whether the checkpoint, serving configuration and operating cost fit your workload.
The release-documentation commit dates to November 19, followed by serving-instruction updates on November 20. This is not the smaller Hy4 Flash, and the repository's earlier creation date is not evidence that today's complete release was available then.
Choose the checkpoint before the hardware
The official deployment guide lists three variants: Hy4 for instruction-following, Hy4-FP8 for quantized serving, and Hy4-base for pretrained-model work. The backbone has 770 billion parameters, with 49 billion active per text token, plus separate vision and draft-model components. Its advertised context window is 1,048,576 tokens.
For a chat evaluation, choose an instruct checkpoint. Comparing a base model's unprompted responses with an instruction-tuned competitor would measure the wrong thing. Record the checkpoint revision alongside every result, rather than labelling all three downloads simply “Hy4.”
The Hugging Face artifact exposes the weights and configuration. Save a manifest of the files you deploy. A model name in an application configuration is not enough to reproduce an evaluation after the repository changes.
Active parameters are not a memory budget
An expert model uses only part of its network for a token, but those active parameters are not the full collection of weights your deployment must accommodate.
As a rough storage calculation, 770 billion values at two bytes each are 1.54 trillion bytes, or about 1.54 TB in decimal units. At one byte each, that becomes roughly 770 GB. These are arithmetic estimates for backbone values, not measured deployment requirements. They exclude other components, quantization metadata, runtime allocations and conversation state.
Prepare a capacity sheet with separate rows for downloaded files, loaded weights, runtime overhead, prompt processing and simultaneous conversations. Do not fill its final row by multiplying the active-parameter count by a precision width.
A large context window also needs its own test. Start with the document sizes customers actually submit, then increase context and concurrency separately. Otherwise an out-of-memory failure tells you little about which dimension exhausted capacity.
Use the model-specific serving path
Tencent's examples use dedicated images, vllm/vllm-openai:hy4 and lmsysorg/sglang:hy4, with the FP8 checkpoint and eight-way tensor parallelism. They include DFly speculative decoding. The vLLM example also selects the hy_v4 reasoning and tool parsers and limits each prompt to one image and no video.
Treat those as launch configurations, not proof that any eight GPUs will work. Check the selected machines' memory and interconnect before booking them. Pin the image digest in your deployment record so a later rebuild does not silently become a different test.
Begin with one short text request. Then test an image request, a long prompt and a tool call independently. If the text path works but structured actions fail, inspect the parser and client handling before changing model weights or sampling settings.
For example, a document assistant should first extract a date from a small page. Only after that succeeds should it attempt a multi-document comparison. This isolates image ingestion from retrieval quality and reasoning.
Test reasoning as a deliberate choice
The release recommends temperature 0.9 and top-p 1.0. Reasoning defaults to high; direct answers use no_think through extra_body.chat_template_kwargs.reasoning_effort.
Run both settings on the same workload rather than assuming deeper reasoning is always the better product default. A support classifier and a code-repair agent have different tolerance for waiting.
Track complete-request latency, output length and task success together. Record whether tool arguments are valid and whether the application executes the intended action. A fluent explanation should not receive credit for an action that never completed.
Keep the initial evaluation read-only. For an invoice assistant, extract fields and propose a categorization without modifying the accounting system. That makes mistakes easier to inspect before adding consequential actions.
Read the actual license
The Tencent Hy License is a modified license, not Apache 2.0. Section 10 adds separate-license conditions for certain large model-as-a-service businesses, with specified exclusions and an internal-use provision. Check the complete terms against your intended service rather than treating downloadable weights as unrestricted commercial permission.
Keep that review separate from technical readiness. A successful server launch answers whether inference runs. It does not settle whether the deployment is appropriate for your business, whether quality is sufficient, or whether maintaining the cluster is cheaper than buying inference.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help AI teams turn model releases and complex deployment choices into technical writing developers can use.