❯ Tencent opens three Hy4 flash variants with image input and a million-token context window
Weights for three usesTencent has released Hy4 flash, Hy4 flash-FP8 and Hy4 flash Base. Developers can choose an application-oriented model, lower-precision weights or a base for further training. Tencent’s model card calls flash its most cost-effective Hy model to date and documents local deployment.
A fraction active per tokenThe model card lists approximately 220 billion total parameters, including about 64 billion Engram parameters. Its mixture-of-experts architecture selects a subset of experts for each token, activating roughly 12 billion parameters. It accepts text and images, produces text and supports a context window of up to about one million tokens.
Why only 16 of 512 experts take a tokenThe model card provides a more useful deployment clue than total size alone. Each token selects 16 of 512 routed experts alongside one shared expert. Engram stores learned representations of common adjacent token combinations for lookup. The complete weights affect what a team must provision to host the model. The path actually taken by each token affects ongoing compute requirements.
What a million-token window allowsA developer could supply a long document containing scanned pages, tables and notes, then ask for a textual answer. The context window governs how much fits in one request. Accuracy still has to be checked against the relevant page, chart and expected answer. With weights at different precisions and documented inference frameworks, teams can benchmark memory use and time to first response on the same examples before choosing between hosted and local service.
Hosted and self-managed optionsDevelopers can also call the model through Tencent Cloud TokenHub. WorkBuddy, Yuanbao, WeCom and QQ have integrated it. Tencent says work is underway to open Hy4 as well. The weights available in this release belong to the flash family.
▮ SIGNALWith the weights available, developers can use the same long documents and images to compare self-hosted Hy4 flash with API access on both cost and results.