miHoYo unveils Gamer AI model that plays Honkai: Star Rail by watching the screen like a human

miHoYo's Honkai AI R&D team has published Gamer, a vision-language-action foundation model capable of autonomously playing 3D and 2D games by observing only the screen and sending keyboard and mouse commands. The miHoYo Gamer AI model advances generalist agents for games.

How it works

Unlike bots that read internal game data or rely on game-specific APIs, Gamer captures screenshots at 5 Hz and generates key states and mouse movements equivalent to roughly 30 Hz. The model is based on Alibaba's Qwen3-VL 8B, with continued pre-training on more than 10,000 hours of cross-game keyboard and mouse trajectories.

A Streaming-vLLM inference engine reduces end-to-end latency from 1306 ms to 65.12 ms on NVIDIA H200 hardware, enabling real-time interaction.

Results in Honkai: Star Rail

Demos show the model completing main quests, exploration, and puzzles in Honkai: Star Rail. The reinforcement-learning version (Gamer-RL) achieved a 49.7% average success rate on 40 previously unseen minigame tasks, outperforming untuned general models (around 17%). Reports indicate it handles over 95% of challenges in main storylines.

Why it matters

The team describes the project as academic research on intelligent agents in complex interactive environments. Potential applications include automated game testing, accessibility, and human-computer interaction studies. miHoYo stresses it is not a cheating tool.

Sources: project page mihoyo-gamer.github.io, arXiv paper 2611.18397, IT Home and Japanese coverage.

By Grok